Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Programming

Sudoku, Linear Optimization, and the Ten Cent Diet

Google Code Blog - 11 hours 47 min ago
Originally posted on the Google Research blog. Cross posted on the Google Apps Developers blog

In 1945, future Nobel laureate George Stigler wrote an essay in the Journal of Farm Economics titled The Cost of Subsistence about a seemingly simple problem: how could a soldier be fed for as little money as possible?

The “Stigler Diet” became a classic problem in the then-new field of linear optimization, which is used today in many areas of science and engineering. Any time you have a set of linear constraints such as “at least 50 square meters of solar panels” or “the amount of paint should equal the amount of primer” along with a linear goal (e.g., “minimize cost” or “maximize customers served”), that’s a linear optimization problem.

At Google, our engineers work on plenty of optimization problems. One example is our YouTube video stabilization system, which uses linear optimization to eliminate the shakiness of handheld cameras. A more lighthearted example is in the Google Docs Sudoku add-on, which instantaneously generates and solves Sudoku puzzles inside a Google Sheet, using the SCIP mixed integer programming solver to compute the solution.
Today we’re proud to announce two new ways for everyone to solve linear optimization problems. First, you can now solve linear optimization problems in Google Sheets with the Linear Optimization add-on written by Google Software Engineer Mihai Amarandei-Stavila. The add-on uses Google Apps Script to send optimization problems to Google servers. The solutions are displayed inside the spreadsheet. For developers who want to create their own applications on top of Google Apps, we also provide an API to let you call our linear solver directly.
Second, we’re open-sourcing the linear solver underlying the add-on: Glop (the Google Linear Optimization Package), created by Bruno de Backer with other members of the Google Optimization team. It’s available as part of the or-tools suite and we provide a few examples to get you started. On that page, you’ll find the Glop solution to the Stigler diet problem. (A Google Sheets file that uses Glop and the Linear Optimization add-on to solve the Stigler diet problem is available here. You’ll need to install the add-on first.)

Stigler posed his problem as follows: given nine nutrients (calories, protein, Vitamin C, and so on) and 77 candidate foods, find the foods that could sustain soldiers at minimum cost.

The Simplex algorithm for linear optimization was two years away from being invented, so Stigler had to do his best, arriving at a diet that cost $39.93 per year (in 1939 dollars), or just over ten cents per day. Even that wasn’t the cheapest diet. In 1947, Jack Laderman used Simplex, nine calculator-wielding clerks, and 120 person-days to arrive at the optimal solution.

Glop’s Simplex implementation solves the problem in 300 milliseconds. Unfortunately, Stigler didn’t include taste as a constraint, and so the poor hypothetical soldiers will eat nothing but the following, ever:

  • Enriched wheat flour
  • Liver
  • Cabbage
  • Spinach
  • Navy beans

Is it possible to create an appealing dish out of these five ingredients? Google Chef Anthony Marco took it as a challenge, and we’re calling the result Foie Linéaire à la Stigler:
This optimal meal consists of seared calf liver dredged in flour, atop a navy bean purée with marinated cabbage and a spinach pesto.

Chef Marco reported that the most difficult constraint was making the dish tasty without butter or cream. That said, I had the opportunity to taste our linear optimization solution, and it was delicious.

Posted by Jon Orwant, Engineering Manager
Categories: Programming

PostgreSQL: ERROR: column does not exist

Mark Needham - Mon, 09/29/2014 - 23:40

I’ve been playing around with PostgreSQL recently and in particular the Northwind dataset typically used as an introductory data set for relational databases.

Having imported the data I wanted to take a quick look at the employees table:

postgres=# SELECT * FROM employees LIMIT 1;
 EmployeeID | LastName | FirstName |        Title         | TitleOfCourtesy | BirthDate  |  HireDate  |           Address           |  City   | Region | PostalCode | Country |   HomePhone    | Extension | Photo |                                                                                      Notes                                                                                      | ReportsTo |              PhotoPath               
------------+----------+-----------+----------------------+-----------------+------------+------------+-----------------------------+---------+--------+------------+---------+----------------+-----------+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+--------------------------------------
          1 | Davolio  | Nancy     | Sales Representative | Ms.             | 1948-12-08 | 1992-05-01 | 507 - 20th Ave. E.\nApt. 2A | Seattle | WA     | 98122      | USA     | (206) 555-9857 | 5467      | \x    | Education includes a BA IN psychology FROM Colorado State University IN 1970.  She also completed "The Art of the Cold Call."  Nancy IS a member OF Toastmasters International. |         2 | http://accweb/emmployees/davolio.bmp
(1 ROW)

That works fine but what if I only want to return the ‘EmployeeID’ field?

postgres=# SELECT EmployeeID FROM employees LIMIT 1;
ERROR:  COLUMN "employeeid" does NOT exist
LINE 1: SELECT EmployeeID FROM employees LIMIT 1;

I hadn’t realised (or had forgotten) that field names get lower cased so we need to quote the name if it’s been stored in mixed case:

postgres=# SELECT "EmployeeID" FROM employees LIMIT 1;
 EmployeeID 
------------
          1
(1 ROW)

From my reading the suggestion seems to be to have your field names lower cased to avoid this problem but since it’s just a dummy data set I guess I’ll just put up with the quoting overhead for now.

Categories: Programming

R: Deriving a new data frame column based on containing string

Mark Needham - Mon, 09/29/2014 - 22:37

I’ve been playing around with R data frames a bit more and one thing I wanted to do was derive a new column based on the text contained in the existing column.

I started with something like this:

> x = data.frame(name = c("Java Hackathon", "Intro to Graphs", "Hands on Cypher"))
> x
             name
1  Java Hackathon
2 Intro to Graphs
3 Hands on Cypher

And I wanted to derive a new column based on whether or not the session was a practical one. The grepl function seemed to be the best tool for the job:

> grepl("Hackathon|Hands on|Hands On", x$name)
[1]  TRUE FALSE  TRUE

We can then add a column to our data frame with that output:

x$practical = grepl("Hackathon|Hands on|Hands On", x$name)

And we end up with the following:

> x
             name practical
1  Java Hackathon      TRUE
2 Intro to Graphs     FALSE
3 Hands on Cypher      TRUE

Not too tricky but it took me a bit too long to figure it out so I thought I’d save future Mark some time!

Categories: Programming

The Future of Jobs

Will you have a job in the future?

What will that job look like and how will the nature of work change?

Will automation take over your job in the near future?

These are the kinds of questions that Ruth Fisher, author of Winning the Hardware-Software Game, has tackled in a series of posts.

I wrote a summary post to distill her big ideas and insights about the future of jobs in my post:

The Future of Jobs

Fisher has done an outstanding job of framing out the landscape and walking the various arguments and perspectives on how automation will change the nature of work and shape the future of jobs.

One of the first things you might be wondering is, what jobs will automation take away?

Fisher addresses that.

Another question is, what new types jobs will be created?

While that’s an exercise for the reader, Fisher provides clues based on what industry luminaries have seen in terms of how jobs are changing.

The key is to know what automation can and can’t do, and to look at the pattern of work in terms of what’s better suited for humans, and what’s better suited for machines.

As one of my mentors puts it, “If the work can be automated, it’s not human.”

He’s a fan of people doing creative, non-routine work, where they can thrive and shine.

As I take on work, or push back on work, I look through a pretty simple lens:

  1. Is the work repetitive in nature? (in which case, something that should be automated)
  2. Is the work a high-value activity? (if not, why am I doing non high-value activities?)
  3. Does the work create greater capability? (for me, the team, the organization, etc.)
  4. Does the work play to my strengths? (if not, who is a better resource or provider.  You grow faster in your strengths, and in today’s world, if people aren’t giving their best where they have their best to give, it leads to a low-impact team that eventually gets out-executed, or put out to Pasteur.)
  5. Does the work lead to world-class impact?  (When everything gets exposed beyond the firewall, and when it’s a globally connected ecosystem, it’s really important to not only bring your A-game, but to play in a way where you can provide the best service in the world for your specific niche.   If you can’t be the best in your niche in a sustainable way, then you’re in the wrong niche.)

I find that by using this simple lens, I tend to take on high-value work that creates high-impact, that cannot be easily automated.  At the same time, while I perform the work, I look for way to turn things into repetitive activities that can be outsources or automated so that I can keep moving up the stack, and producing higher-value work … that’s more human.

Categories: Architecture, Programming

What Ever Happened to the Founders of Sierra Online?

Making the Complex Simple - John Sonmez - Mon, 09/29/2014 - 15:00

Some of the fondest memories of my childhood involve playing adventure games like Space Quest, Kings Quest and Quest for Glory. I remember spending countless hours reloading from save spots and trying to figure out a puzzle. I remember that exciting feeling of anticipation when the Sierra logo flashed onto the screen as my 486 […]

The post What Ever Happened to the Founders of Sierra Online? appeared first on Simple Programmer.

Categories: Programming

R: Filtering data frames by column type (‘x’ must be numeric)

Mark Needham - Mon, 09/29/2014 - 06:46

I’ve been working through the exercises from An Introduction to Statistical Learning and one of them required you to create a pair wise correlation matrix of variables in a data frame.

The exercise uses the ‘Carseats’ data set which can be imported like so:

> install.packages("ISLR")
> library(ISLR)
> head(Carseats)
  Sales CompPrice Income Advertising Population Price ShelveLoc Age Education Urban  US
1  9.50       138     73          11        276   120       Bad  42        17   Yes Yes
2 11.22       111     48          16        260    83      Good  65        10   Yes Yes
3 10.06       113     35          10        269    80    Medium  59        12   Yes Yes
4  7.40       117    100           4        466    97    Medium  55        14   Yes Yes
5  4.15       141     64           3        340   128       Bad  38        13   Yes  No
6 10.81       124    113          13        501    72       Bad  78        16    No Yes

filter the categorical variables from a data frame and

If we try to run the ‘cor‘ function on the data frame we’ll get the following error:

> cor(Carseats)
Error in cor(Carseats) : 'x' must be numeric

As the error message suggests, we can’t pass non numeric variables to this function so we need to remove the categorical variables from our data frame.

But first we need to work out which columns those are:

> sapply(Carseats, class)
      Sales   CompPrice      Income Advertising  Population       Price   ShelveLoc         Age   Education 
  "numeric"   "numeric"   "numeric"   "numeric"   "numeric"   "numeric"    "factor"   "numeric"   "numeric" 
      Urban          US 
   "factor"    "factor"

We can see a few columns of type ‘factor’ and luckily for us there’s a function which will help us identify those more easily:

> sapply(Carseats, is.factor)
      Sales   CompPrice      Income Advertising  Population       Price   ShelveLoc         Age   Education 
      FALSE       FALSE       FALSE       FALSE       FALSE       FALSE        TRUE       FALSE       FALSE 
      Urban          US 
       TRUE        TRUE

Now we can remove those columns from our data frame and create the correlation matrix:

> cor(Carseats[sapply(Carseats, function(x) !is.factor(x))])
                  Sales   CompPrice       Income  Advertising   Population       Price          Age    Education
Sales        1.00000000  0.06407873  0.151950979  0.269506781  0.050470984 -0.44495073 -0.231815440 -0.051955242
CompPrice    0.06407873  1.00000000 -0.080653423 -0.024198788 -0.094706516  0.58484777 -0.100238817  0.025197050
Income       0.15195098 -0.08065342  1.000000000  0.058994706 -0.007876994 -0.05669820 -0.004670094 -0.056855422
Advertising  0.26950678 -0.02419879  0.058994706  1.000000000  0.265652145  0.04453687 -0.004557497 -0.033594307
Population   0.05047098 -0.09470652 -0.007876994  0.265652145  1.000000000 -0.01214362 -0.042663355 -0.106378231
Price       -0.44495073  0.58484777 -0.056698202  0.044536874 -0.012143620  1.00000000 -0.102176839  0.011746599
Age         -0.23181544 -0.10023882 -0.004670094 -0.004557497 -0.042663355 -0.10217684  1.000000000  0.006488032
Education   -0.05195524  0.02519705 -0.056855422 -0.033594307 -0.106378231  0.01174660  0.006488032  1.000000000
Categories: Programming

1 Week in F#

Phil Trelford's Array - Sun, 09/28/2014 - 20:30

With Sergey Tihon on vacation this week, I’ve collated a one off alternative F# Weekly roundup, covering some of the highlights from another busy week in the F# community.

News in brief

FSharp Logo

Events

This week has seen meetups in Nashville, Portland, Washington DC, Stockholm and London.

The Multi-hit brick! #fsharp #gamedev #breakout @DCFSharp @silverSpoon pic.twitter.com/nVPxUrhIZg

— Wesley Wiser (@wesleywiser) September 25, 2014

Recordings

Upcoming meetups

Upcoming Conferences

Projects

Blogs

FsiBot

.@djidja8 " ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ▤ ⬛ ▤ ⬛ ♙ ⬛ ▤ ⬛ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ⬛ ▤ ⬛ ▤ ⬛ ▤ ⬛ ▤"

— fsibot (@fsibot) September 22, 2014

Have a great week!

Categories: Programming

Dazzle Your Audience By Doodling

Xebia Blog - Sun, 09/28/2014 - 10:29

When we were kids, we loved to doodle. Most of us did anyway. I doodled all the time, everywhere, and, to the dismay of my mother, on everything. I still love to doodle. In fact, I believe doodling is essential.

The tragedy of the doodle lies in its definition: "A doodle is an unfocused or unconscious drawing while a person's attention is otherwise occupied." That's why most of us have been taught not to doodle. Seems logical, right? Teacher sees you doodling, that is not paying attention in class, thus not learning as much as you should, so he puts a stop to it. Trouble is though, it's wrong. And it's not just a little bit wrong, it's totally and utterly wrong. Exactly how wrong was shown in a case study by Jackie Andrade. She discovered that doodlers have 29% better recall. So, if you don't doodle, you're doing yourself a disservice.

And you're not just doing yourself a disservice, you're also doing your audience a disservice. Neurologists have discovered a phenomenon dubbed "mirror neurons." When you see something, the same neurons fire as if you were doing it. So, if someone shows you a picture, let's say a slide in a presentation, it is as if you're showing that picture to yourself.

Wait, what? That doesn't sound special at all, now does it? That's why presentations using only slides can be so unintentionally relaxing.

Now, if you see someone write or draw something on a flip chart, dry erase board or any other surface in plain sight, it is as if you're writing or drawing it yourself. And that ensures 29% better recall. Better yet, you'll remember what the presenter wants you to rememeber. Especially if he can trigger an emotional response.

Now, why is that? At EUVIZ in Berlin last month, I attended a presentation by Barbara Siegel from Look2Listen that changed my life. Barbara talked about the latest insights from neuroscience that prove that everyone feels first and thinks later. So, if you want your audience to tune in to your talk, show some emotion! Want people to remember specific points of your talk? Trigger and capture emotion by writing and drawing in real-time. Emotion runs deep and draws firm neurological paths in the brain that help you recreate the memory. Memories are recreated, not stored and retrieved.

Another thing that helps you draw firm neurological paths is exercise. If you get your audience to stand up and move, you increase their brain activity by 7%, hightening alertness and motivation. By getting your audience to sit down again after physical exercise, you trigger a rebalancing of neurotransmitters and other neurochemicals, so they can use the newly spawned neurons in their brain to combine into memories of your talk. Now that got me running every other day! Well, jogging is more like it, but hey: I'm hitting my target heart-rate regularly!

How does this help you become a better public speaker? Remember these two key points:

  1. At the start of your speech, get your audience to stand up and move to ensure 7% more brain activity and prime them for maximum recall.
  2. Make sure to use visuals and metaphors and create most, if not all, of them in real-time to leverage the mirror neuron effect and increase recall by 29%.

Neo4j: COLLECTing multiple values (Too many parameters for function ‘collect’)

Mark Needham - Fri, 09/26/2014 - 21:46

One of my favourite functions in Neo4j’s cypher query language is COLLECT which allows us to group items into an array for later consumption.

However, I’ve noticed that people sometimes have trouble working out how to collect multiple items with COLLECT and struggle to find a way to do so.

Consider the following data set:

create (p:Person {name: "Mark"})
create (e1:Event {name: "Event1", timestamp: 1234})
create (e2:Event {name: "Event2", timestamp: 4567})
 
create (p)-[:EVENT]->(e1)
create (p)-[:EVENT]->(e2)

If we wanted to return each person along with a collection of the event names they’d participated in we could write the following:

$ MATCH (p:Person)-[:EVENT]->(e)
> RETURN p, COLLECT(e.name);
+--------------------------------------------+
| p                    | COLLECT(e.name)     |
+--------------------------------------------+
| Node[0]{name:"Mark"} | ["Event1","Event2"] |
+--------------------------------------------+
1 row

That works nicely, but what about if we want to collect the event name and the timestamp but don’t want to return the entire event node?

An approach I’ve seen a few people try during workshops is the following:

MATCH (p:Person)-[:EVENT]->(e)
RETURN p, COLLECT(e.name, e.timestamp)

Unfortunately this doesn’t compile:

SyntaxException: Too many parameters for function 'collect' (line 2, column 11)
"RETURN p, COLLECT(e.name, e.timestamp)"
           ^

As the error message suggests, the COLLECT function only takes one argument so we need to find another way to solve our problem.

One way is to put the two values into a literal array which will result in an array of arrays as our return result:

$ MATCH (p:Person)-[:EVENT]->(e)
> RETURN p, COLLECT([e.name, e.timestamp]);
+----------------------------------------------------------+
| p                    | COLLECT([e.name, e.timestamp])    |
+----------------------------------------------------------+
| Node[0]{name:"Mark"} | [["Event1",1234],["Event2",4567]] |
+----------------------------------------------------------+
1 row

The annoying thing about this approach is that as you add more items you’ll forget in which position you’ve put each bit of data so I think a preferable approach is to collect a map of items instead:

$ MATCH (p:Person)-[:EVENT]->(e)
> RETURN p, COLLECT({eventName: e.name, eventTimestamp: e.timestamp});
+--------------------------------------------------------------------------------------------------------------------------+
| p                    | COLLECT({eventName: e.name, eventTimestamp: e.timestamp})                                         |
+--------------------------------------------------------------------------------------------------------------------------+
| Node[0]{name:"Mark"} | [{eventName -> "Event1", eventTimestamp -> 1234},{eventName -> "Event2", eventTimestamp -> 4567}] |
+--------------------------------------------------------------------------------------------------------------------------+
1 row

During the Clojure Neo4j Hackathon that we ran earlier this week this proved to be a particularly pleasing approach as we could easily destructure the collection of maps in our Clojure code.

Categories: Programming

Tell us about your experience building on Google, and raise money for educational organizations!

Google Code Blog - Thu, 09/25/2014 - 22:16
Here at Google, we always put the user first, and for the Developer Platform team, our developers are our users. We want to create the best development platform and provide the support you need to build world-changing apps, but we need to hear from you, our users, on a regular basis so we can see what’s working and what needs to change.

That's why we're launching our developer survey -- we want to hear about how you are using our APIs and platforms, and what your experience is using our developer products and services. We'll use your responses to identify how we can support you better in your development efforts.
Photo Credit: Google I/O 2014

The survey should only take 10 to 15 minutes of your time, and in addition to helping us improve our products, you can also help raise money to educate children around the globe. For every developer who completes the survey, we will donate $10 USD (up to a maximum amount of $20,000USD total) to your choice of one of these six education-focused organizations: Khan Academy, World Fund, Donors Choose, Girl Rising, Raspberry Pi, and Agastya.

The survey is live now and will be live until 11:59PM Pacific Time on October 15, 2014. We are excited to hear what you have to tell us!

Posted by Neel Kshetramade, Program Manager, Developer Platform
Categories: Programming

Allthecooks on Android Wear

Android Developers Blog - Wed, 09/24/2014 - 21:56

By Hoi Lam, Developer Advocate, Android Wear

The best cooking companion since the apron?

Android Wear is designed for serving up useful information at just the right time and in the right place. A neat example of this is Allthecooks Recipes. It gives you the right recipe, right when you need it.

This app is a great illustration of the four creative visions for Android Wear:

  1. Launched automatically
  2. Glanceable
  3. Suggest and demand
  4. Zero or low interaction

Allthecooks also shows what developers can do by combining both the power of the mobile device and the convenience of Android Wear.

Pick the best tool for the job

One particularly well-designed aspect of Allthecooks is their approach to the multi-device experience. Allthecooks lets the user search and browse the different recipes on their Android phone or tablet. When the user is ready, there is a clearly labelled blue action link to send the recipe to the watch.

The integration is natural. Using the on-screen keyboard and the larger screen real estate, Allthecooks is using the best screen to browse through the recipes. On the wearables side, the recipe is synchronised by using the DataApi and is launched automatically, fulfilling one of the key creative visions for Android Wear.

The end result? The mobile / Wear integration is seamless.

Thoughtful navigation

Once the recipe has been sent to the Android Wear device, Allthecooks splits the steps into easily glanceable pages. At the end of that list of steps, it allows the user to jump back to the beginning with a clearly marked button.

This means if you would like to browse through the steps before starting to cook, you can effortlessly get to the beginning again without swiping through all the pages. This is a great example of two other points in the vision: glanceable and zero or low interaction.

A great (cooking) assistant

One of the key ingredients of great cooking is timing, and Allthecooks is always on hand to do all the inputs for you when you are ready to start the clock. A simple tap on the blue “1” and Allthecooks will automatically set the timer to one hour. It is a gentle suggestion that Allthecooks can set the timer for you if you want.

Alternatively, if you want to use your egg timer, why not? It is a small detail but it really demonstrates the last and final element of Android Wear’s vision of suggest and demand. It is an ever ready assistant when the user wants it. At the same time, it is respectful and does not force the user to go down a route that the user does not want.

It’s about the details

Great design is about being user-centric and paying attention to details. Allthecooks could have just shrunk their mobile app for wear. Instead the Allthecooks team put a lot of thoughts into the design and leveraged all four points of the Android Wear creative vision. The end result is that the user can get the best experience out of both their Android mobile device and their Android Wear device. So developers, what will you be cooking next on Android Wear?

For more inspiring Android Wear user experiences, check out the Android Wear collection on Google Play!


Join the discussion on
+Android Developers


Categories: Programming

Neo4j: LOAD CSV – Column is null

Mark Needham - Wed, 09/24/2014 - 21:21

One problem I’ve seen a few people have recently when using Neo4j’s LOAD CSV function is dealing with CSV files that have dodgy hidden characters at the beginning of the header line.

For example, consider an import of this CSV file:

$ cat ~/Downloads/dodgy.csv
userId,movieId
1,2

We might start by checking which columns it has:

$ load csv with headers from "file:/Users/markneedham/Downloads/dodgy.csv" as line return line;
+----------------------------------+
| line                             |
+----------------------------------+
| {userId -> "1", movieId -> "2"} |
+----------------------------------+
1 row

Looks good so far but what about if we try to return just ‘userId’?

$ load csv with headers from "file:/Users/markneedham/Downloads/dodgy.csv" as line return line.userId;
+-------------+
| line.userId |
+-------------+
| <null>      |
+-------------+
1 row

Hmmm it’s null…what about ‘movieId’?

$ load csv with headers from "file:/Users/markneedham/Downloads/dodgy.csv" as line return line.movieId;
+--------------+
| line.movieId |
+--------------+
| "2"          |
+--------------+
1 row

That works fine so immediately we can suspect there are hidden characters at the beginning of the first line of the file.

The easiest way to check if this is the case is open the file using a Hex Editor – I quite like Hex Fiend for the Mac.

If we look at dodgy.csv we’ll see the following:

2014 09 24 21 20 06

Let’s delete the highlighted characters and try our cypher query again:

$ load csv with headers from "file:/Users/markneedham/Downloads/dodgy.csv" as line return line.userId;
+-------------+
| line.userId |
+-------------+
| "1"         |
+-------------+
1 row

All is well again, but something to keep in mind if you see a LOAD CSV near you behaving badly.

Categories: Programming

New Google Apps Activity API

Google Code Blog - Wed, 09/24/2014 - 18:21
Back in January, Google Drive launched an activity stream that shows you what actions have been taken on files and folders in your Drive. For example, if someone makes edits on a file you’ve shared with them, you’ll see a notification in your activity stream.
Today, we’re introducing the new Google Apps Activity API designed to give developers programmatic access to this activity stream. This standard Google API will allow apps and extensions to access the activity history for individual Drive files as well as descendents of a folder through a RESTful interface.
The Google Apps Activity API will allow developers to build new tools to help users keep better track of what’s happening to specific files and folders they care about. For example, you might use this new API to help teachers see which students in their class are editing a file or, come tax season, you might want to create a quick script to audit the sharing of items in your financial information folder.
Check out the documentation at https://developers.google.com/google-apps/activity/. We can't wait to see what you build!
Posted by Justin Hicks, Software Engineer, Technical Lead for Google Apps Activity API
Categories: Programming

New features in Admin SDK: Custom user attributes, and opening up access to all domain users

Google Code Blog - Tue, 09/23/2014 - 19:21
By Muzammil Esmail, Product Manager, Google for Work
The Admin SDK provides a comprehensive directory experience for Google for Work customers to help them meet specific business needs around data storage for customers. Here are some important updates to this SDK.
Custom attributes in the user’s profileNow available is a new feature in the Directory API which allows you to add custom attributes for your users. For instance, you could store the projects your users work on, their desk number, job level, hiring date — whatever makes sense for your business.
Once the custom attributes for your domain have been defined, they behave just like regular fields in the user profile. You can get and set them for your users and also perform searches on custom fields (e.g. “all employees that work on the shinyNewApp in Hyderabad”).
Custom attributes can be of different data types; they can be single- or multi-valued. You can configure whether they are “public” i.e. visible to everyone on the domain, or “private” i.e. visible only to admins and the users themselves.
Read access to all domain usersHistorically, only admins have been able to access the data in the Admin SDK. Beginning today, any user (not just admins) will now be able to call the Directory API to read the profile of any user on the domain (of course, we will respect ACLing settings and profile sharing settings).
We hope that you will be able to use this new feature to build business applications (e.g. corporate yellow pages, expense approval, vacation management, workflow applications, etc.) that can be used by all your users.
Please feel free to go through our documentation to go learn more about the Admin SDK, and specifically the Directory API. Happy hacking!
Categories: Programming

Conference Data Sync and GCM in the Google I/O App

Android Developers Blog - Tue, 09/23/2014 - 18:44
By Bruno Oliveira, tech lead of the 2014 Google I/O mobile app

Keeping data in sync with the cloud is an important part of many applications, and the Google I/O App is no exception. To do this, we leverage the standard Android mechanism for this purpose: a Sync Adapter. Using a Sync Adapter has many benefits over using a more rudimentary mechanism such as setting up recurring alarms, because the system automatically handles the scheduling of Sync Adapters to optimize battery life.

We store the data in a local SQLite database. However, rather than having the whole application access that database directly, the application employs another standard Android mechanism to control and organize access to that data. This structure is, naturally, a Content Provider. Only the content provider's implementation has direct access to the SQLite database. All other parts of the app can only access data through the Content Resolver. This allows for a very flexible decoupling between the representation of the data in the database and the more abstract view of that data that is used throughout the app.

The I/O app maintains with two main kinds of data: conference data (sessions, speakers, rooms, etc) and user data (the user's personalized schedule). Conference data is kept up to date with a one-way sync from a set of JSON files stored in Google Cloud Storage, whereas user data goes through a two-way sync with a file stored in the user's Google Drive AppData folder.

Downloading Conference Data Efficiently

For a conference like Google I/O, conference data can be somewhat large. It consists of information about all the sessions, rooms, speakers, map locations, social hashtags, video library items and others. Downloading the whole data set repeatedly would be wasteful both in terms of battery and bandwidth, so we adopt a strategy to minimize the amount of data we download and process.

This strategy is separating the data into several different JSON files, and having them be referenced by a central master JSON file called the manifest file. The URL of the manifest file is the only URL that is hard-coded into the app (it is defined by the MANIFEST_URL constant in Config.java). Note that the I/O app uses Google Cloud Storage to store and serve these files, but any robust hosting service accessible via HTTP can be used for the same purpose.

The first part of the sync process is checking if the manifest file was changed since the app last downloaded it, and processing it only if it's newer. This logic is implemented by the fetchConfenceDataIfNewer method in RemoteConferenceDataFetcher.

public class RemoteConferenceDataFetcher {
    // (...)
    public String[] fetchConferenceDataIfNewer(String refTimestamp) throws IOException {
        BasicHttpClient httpClient = new BasicHttpClient();
        httpClient.setRequestLogger(mQuietLogger);
        // (...)

        // Only download if data is newer than refTimestamp
        if (!TextUtils.isEmpty(refTimestamp) && TimeUtils
            .isValidFormatForIfModifiedSinceHeader(refTimestamp)) {
                httpClient.addHeader("If-Modified-Since", refTimestamp);
            }
        }

        HttpResponse response = httpClient.get(mManifestUrl, null);
        int status = response.getStatus();
        if (status == HttpURLConnection.HTTP_OK) {
            // Data modified since we last checked -- process it!
        } else if (status == HttpURLConnection.HTTP_NOT_MODIFIED) {
            // data on the server is not newer than our data - no work to do!
            return null;
        } else {
            // (handle error)
        }
    }
    // (...)
}

Notice that we submit the HTTP If-Modified-Since header with our request, so that if the manifest hasn't changed since we last checked it, we will get an HTTP response code of HTTP_NOT_MODIFIED rather than HTTP_OK, we will react by skipping the download and parsing process. This means that unless the manifest has changed since we last saw it, the sync process is very economical: it consists only of a single HTTP request and a short response.

The manifest file's format is straightforward: it consists of references to other JSON files that contain the relevant pieces of the conference data:

{
  "format": "iosched-json-v1",
  "data_files": [
    "past_io_videolibrary_v5.json",
    "experts_v11.json",
    "hashtags_v8.json",
    "blocks_v10.json",
    "map_v11.json",
    "keynote_v10.json",
    "partners_v2.json",
    "session_data_v2.681.json"
  ]
}

The sync process then proceeds to process each of the listed data files in order. This part is also implemented to be as economical as possible: if we detect that we already have a cached version of a specific data file, we skip it entirely and use our local cache instead. This task is done by the processManifest method.

Then, each JSON file is parsed and the entities present in each one are accumulated in memory. At the end of this process, the data is written to the Content Provider.

Issuing Content Provider Operations Efficiently

The conference data sync needs to be efficient not only in the amount of data it downloads, but also in the amount of operations it performs on the database. This must be done as economically as possible, so this step is also optimized: instead of overwriting the whole database with the new data, the Sync Adapter attempts to preserve the existing entities and only update the ones that have changed. In our tests, this optimization step reduced the total sync time from 16 seconds to around 2 seconds on our test devices.

In order to accomplish this important third layer of optimization, the application needs to know, given an entity in memory and its version in the Content Provider, whether or not we need to issue content provider operations to update that entity. Comparing the entity in memory to the entity in the database field by field is one option, but is cumbersome and slow, since it would require us to read every field. Instead, we add a field to each entity called the import hashcode. The import hashcode is a weak hash value generated from its data. For example, here is how the import hashcode for a speaker is computed:

public class Speaker {
    public String id;
    public String publicPlusId;
    public String bio;
    public String name;
    public String company;
    public String plusoneUrl;
    public String thumbnailUrl;

    public String getImportHashcode() {
        StringBuilder sb = new StringBuilder();
        sb.append("id").append(id == null ? "" : id)
                .append("publicPlusId")
                .append(publicPlusId == null ? "" : publicPlusId)
                .append("bio")
                .append(bio == null ? "" : bio)
                .append("name")
                .append(name == null ? "" : name)
                .append("company")
                .append(company== null ? "" : company)
                .append("plusoneUrl")
                .append(plusoneUrl == null ? "" : plusoneUrl)
                .append("thumbnailUrl")
                .append(thumbnailUrl == null ? "" : thumbnailUrl);
        String result = sb.toString();
        return String.format(Locale.US, "%08x%08x", 
            result.hashCode(), result.length());
    }
}

Every time an entity is updated in the database, its import hashcode is saved with it as a database column. Later, when we have a candidate for an updated version of that entity, all we need to do is compute the import hashcode of the candidate and compare it to the import hashcode of the entity in the database. If they differ, then we issue Content Provider operations to update the entity in the database. If they are the same, we skip that entity. This incremental update logic can be seen, for example, in the makeContentProviderOperations method of the SpeakersHandler class:

public class SpeakersHandler extends JSONHandler {
    private HashMap mSpeakers = new HashMap();

    // (...)
    @Override
    public void makeContentProviderOperations(ArrayList list) {
        // (...)
        int updatedSpeakers = 0;
        for (Speaker speaker : mSpeakers.values()) {
            String hashCode = speaker.getImportHashcode();
            speakersToKeep.add(speaker.id);

            if (!isIncrementalUpdate || !speakerHashcodes.containsKey(speaker.id) ||
                    !speakerHashcodes.get(speaker.id).equals(hashCode)) {
                // speaker is new/updated, so issue content provider operations
                ++updatedSpeakers;
                boolean isNew = !isIncrementalUpdate || 
                    !speakerHashcodes.containsKey(speaker.id);
                buildSpeaker(isNew, speaker, list);
            }
        }

        // delete obsolete speakers
        int deletedSpeakers = 0;
        if (isIncrementalUpdate) {
            for (String speakerId : speakerHashcodes.keySet()) {
                if (!speakersToKeep.contains(speakerId)) {
                    buildDeleteOperation(speakerId, list);
                    ++deletedSpeakers;
                }
            }
        }
    }
}

The buildSpeaker and buildDeleteOperation methods (omitted here for brevity) simply build the Content Provider operations necessary to, respectively, insert/update a speaker or delete a speaker from the Content Provider. Notice that this approach means we only issue Content Provider operations to update a speaker if the import hashcode has changed. We also deal with obsolete speakers, that is, speakers that were in the database but were not referenced by the incoming data, and we issue delete operations for those speakers.

Making Sync Robust

The sync adapter in the I/O app is responsible for several tasks, amongst which are the remote conference data sync, the user schedule sync and also the user feedback sync. Failures can happen in any of them because of network conditions and other factors. However, a failure in one of the tasks should not impact the execution of the other tasks. This is why we structure the sync process as a series of independent tasks, each protected by a try/catch block, as can be seen in the performSync method of the SyncHelper class:

// remote sync consists of these operations, which we try one by one (and
// tolerate individual failures on each)
final int OP_REMOTE_SYNC = 0;
final int OP_USER_SCHEDULE_SYNC = 1;
final int OP_USER_FEEDBACK_SYNC = 2;

int[] opsToPerform = userDataOnly ?
        new int[] { OP_USER_SCHEDULE_SYNC } :
        new int[] { OP_REMOTE_SYNC, OP_USER_SCHEDULE_SYNC, OP_USER_FEEDBACK_SYNC};

for (int op : opsToPerform) {
    try {
        switch (op) {
            case OP_REMOTE_SYNC:
                dataChanged |= doRemoteSync();
                break;
            case OP_USER_SCHEDULE_SYNC:
                dataChanged |= doUserScheduleSync(account.name);
                break;
            case OP_USER_FEEDBACK_SYNC:
                doUserFeedbackSync();
                break;
        }
    } catch (AuthException ex) {
        // (... handle auth error...)
    } catch (Throwable throwable) {
        // (... handle other error...)

        // Let system know an exception happened:
        if (syncResult != null && syncResult.stats != null) {
            ++syncResult.stats.numIoExceptions;
        }
    }
}

When one particular part of the sync process fails, we let the system know about it by increasing syncResult.stats.numIoExceptions. This will cause the system to retry the sync at a later time, using exponential backoff.

When Should We Sync? Enter GCM.

It's very important for users to be able to get updates about conference data in a timely manner, especially during (and in the few days leading up to) Google I/O. A naïve way to solve this problem is simply making the app poll the server repeatedly for updates. Naturally, this causes problems with bandwidth and battery consumption.

To solve this problem in a more elegant way, we use GCM (Google Cloud Messaging). Whenever there is an update to the data on the server side, the server sends a GCM message to all registered devices. Upon receipt of this GCM message, the device performs a sync to download the new conference data. The GCMIntentService class handles the incoming GCM messages:

Update (23 September 2014): Since this blog post was first published, the GCMBaseIntentService class has been deprecated. Please use the GoogleCloudMessaging API instead.

public class GCMIntentService extends GCMBaseIntentService {
    private static final String TAG = makeLogTag("GCM");

    private static final Map MESSAGE_RECEIVERS;
    static {
        // Known messages and their GCM message receivers
        Map  receivers = new HashMap();
        receivers.put("test", new TestCommand());
        receivers.put("announcement", new AnnouncementCommand());
        receivers.put("sync_schedule", new SyncCommand());
        receivers.put("sync_user", new SyncUserCommand());
        receivers.put("notification", new NotificationCommand());
        MESSAGE_RECEIVERS = Collections.unmodifiableMap(receivers);
    }

    // (...)

    @Override
    protected void onMessage(Context context, Intent intent) {
        String action = intent.getStringExtra("action");
        String extraData = intent.getStringExtra("extraData");
        LOGD(TAG, "Got GCM message, action=" + action + ", extraData=" + extraData);

        if (action == null) {
            LOGE(TAG, "Message received without command action");
            return;
        }

        action = action.toLowerCase();
        GCMCommand command = MESSAGE_RECEIVERS.get(action);
        if (command == null) {
            LOGE(TAG, "Unknown command received: " + action);
        } else {
            command.execute(this, action, extraData);
        }

    }
    // (...)
}

Notice that the onMessage method delivers the message to the appropriate handler depending on the GCM message's "action" field. If the action field is "sync_schedule", the application delivers the message to an instance of the SyncCommand class, which causes a sync to happen. Incidentally, notice that the implementation of the SyncCommand class allows the GCM message to specify a jitter parameter, which causes it to trigger a sync not immediately but at a random time in the future within the jitter interval. This spreads out the syncs evenly over a period of time rather than forcing all clients to sync simultaneously, and thus prevents a sudden peak in requests on the server side.

Syncing User Data

The I/O app allows the user to build their own personalized schedule by choosing which sessions they are interested in attending. This data must be shared across the user's Android devices, and also between the I/O website and Android. This means this data has to be stored in the cloud, in the user's Google account. We chose to use the Google Drive AppData folder for this task.

User data is synced to Google Drive by the doUserScheduleSync method of the SyncHelper class. If you dive into the source code, you will notice that this method essentially accesses the Google Drive AppData folder through the Google Drive HTTP API, then reconciles the set of sessions in the data with the set of sessions starred by the user on the device, and issues the necessary modifications to the cloud if there are locally updated sessions.

This means that if the user selects one session on their Android device and then selects another session on the I/O website, the result should be that both the Android device and the I/O website will show that both sessions are in the user's schedule.

Also, whenever the user adds or removes a session on the I/O website, the data on all their Android devices should be updated, and vice versa. To accomplish that, the I/O website sends our GCM server a notification every time the user makes a change to their schedule; the GCM server, in turn, sends a GCM message to all the devices owned by that user in order to cause them to sync their user data. The same mechanism works across the user's devices as well: when one device updates the data, it issues a GCM message to all other devices.

Conclusion

Serving fresh data is a key component of many Android apps. This article showed how the I/O app deals with the challenges of keeping the data up-to-date while minimizing network traffic and database changes, and also keeping this data in sync across different platforms and devices through the use of Google Cloud Storage, Google Drive and Google Cloud Messaging.

Categories: Programming

The importance of knowing when you are wrong as an Agile and #NoEstimates principle

Software Development Today - Vasco Duarte - Tue, 09/23/2014 - 04:00

You started the project. You spent hours, no: days! estimating the project. The project starts and your confidence in its success is high.

Everything goes well at the start, but at some point you find the project is late. What happened? How can you be wrong about estimates?

This story very common in software projects. So common, that I bet you have lived through it many times in your life. I know I have!

Let’s get over it. We’re always wrong about estimation. Sometimes more, sometimes less and very, very rarely we are wrong in a way that makes us happy: we overestimated something and can deliver the project ahead of (the inflated?) schedule.

We’re always wrong about estimation.

Being wrong about estimates is the status quo. Get over it. Now let’s take advantage of being wrong! You can save the project by being wrong. Here’s why...

The art of being wrong about software estimates

Knowing you are wrong about your estimates is not difficult after the fact, when you compare estimates to actuals. The difficult part is to make a prediction in a way that can tested regularly, and very early on - when you still have time to change the project.

Software project estimates as they are usually done, delay the feedback for the “on time” performance to a point in time when there’s very little we can do about it. Goldratt grasped this problem and made a radical suggestion: cut all estimates in half, and use the rest of the time as a project buffer. Pretty crazy hein? Well, it worked because it forced projects to face their failures much earlier than they would otherwise. Failing to meet a deadline early on in the life-cycle of the project gave them a very powerful tool in project management: time to react!

The #NoEstimates approach to being wrong...and learning from it

In this video I explain shortly how I make predictions about a possible release date for the project based on available data. Once I make a release date prediction, I validate it as soon as possible, and typically every week. This approach allows me to learn early enough when I’m wrong and then adjust the project as needed.

We’re always wrong, the important thing is to find out how wrong, as early as possible

After each delivery (whether it is a feature or a timebox like a sprint), I update my prediction for the release date of the project based on the lead time or throughput rate so far. After updating the release date projection, I can see whether it has changed enough to require a reaction by the project team. I can make this update to the project schedule without gathering the whole team (or "the chosen ones") into a room for an ungodly long estimation meeting.

If the date has not changed outside the originally interval, or if the delivery rate is stable (see the video), then I don’t need to react.

When the release date projection changes to a time outside the original interval, or the throughput rate has become unstable (did you see the video?), then you need to react. At first to investigate the situation, and later to adjust the parameters in your project if needed.

Conclusion

The #NoEstimates approach I advocate will allow you to know when the project has changed enough to warrant a reaction. I make a prediction, and (at least) every week I review that prediction and take action.

Estimates, done the traditional way, also give you this information, but too late. This happens because of the big-batch thinking the reliance on estimations enables (larger work items are ok if you estimate), and because of the delayed dependency integration it enables (estimated projects typically allow for teams that are dependent to work separately because of the agreed plan).

The #NoEstimates approach I advocate has one goal: reduce feedback cycle. These short feedback cycles will allow you to recognise early enough how wrong you were about your predictions, and then you can make the necessary adjustments!

Picture credit: John Hammink, follow him on twitter

New D-Series of Azure VMs with 60% Faster CPUs, More Memory and Local SSD Disks

ScottGu's Blog - Scott Guthrie - Mon, 09/22/2014 - 19:19

Today I’m excited to announce that we just released a new set of VM sizes for Microsoft Azure. These VM sizes are now available to be used immediately by every Azure customer.

The new D-Series of VMs can be used with both Azure Virtual Machines and Azure Cloud Services.  In addition to offering faster vCPUs (approximately 60% faster than our A series) and more memory (up to 112 GB), the new VM sizes also all have a local SSD disk (up to 800 GB) to enable much faster IO reads and writes.

The new VM sizes available today include the following:

General Purpose D-Series VMs

Name vCores Memory (GB) Local SSD Disk (GB) Standard_D1 1 3.5 50 Standard_D2 2 7 100 Standard_D3 4 14 200 Standard_D4 8 28 400

 

High Memory D-Series VMs

Name vCores Memory (GB) Local SSD Disk (GB) Standard_D11 2 14 100 Standard_D12 4 28 200 Standard_D13 8 56 400 Standard_D14 16 112 800

For pricing information, please see Virtual Machine Pricing Details.

Local SSD Disk and SQL Server Buffer Pool Extensions

A temporary drive on the VMs (D:\ on Windows, /mnt or /mnt/resource on Linux) is mapped to the local SSDs exposed on the D-Service VMs, and provides a really good option for replicated storage workloads, like MongoDB, or for significantly increasing the performance of SQL Server 2014 by enabling its unique Buffer Pool Extensions (BPE) feature.

SQL Server 2014’s Buffer Pool Extensions allows you to extend the SQL Engine Buffer Pool with the memory of local SSD disks to significantly improve the performance of SQL workloads. The Buffer Pool is a global memory resource used to cache data pages for much faster read operations.  Without any code changes in your application, you can enable the buffer pool support with the SSDs of the D-Series VMs using a simple T-SQL query with just four lines:

ALTER SERVER CONFIGURATION
SET BUFFER POOL EXTENSION ON
SIZE = <size> [ KB | MB | GB ]
FILENAME = 'D:\SSDCACHE\EXAMPLE.BPE'

No code changes are required in your application, and all write operations will continue to be durably persisted in VM drives persisted in Azure Storage. More details on configuring and using BPE can be found here.

Start Using the D-Series VMs Today

You can start using the new D-Series VM sizes immediately.  They can be easily created and used via both the current Azure Management Portal as well as Preview Portal, as well as from the Azure management command-line/scripts/APIs.

To learn more about the D-Series please read this post which has even more details about them, as well as check out the Azure documentation center.

Hope this helps,

Scott

Categories: Architecture, Programming

Xebia KnowledgeCast Episode 4: Scrum Day Europe 2013, OpenSpace Knowledge Exchange, and Fun With Stickies!

Xebia Blog - Mon, 09/22/2014 - 16:44

xebia_xkc_podcast
The Xebia KnowledgeCast is a bi-weekly podcast about software architecture, software development, lean/agile, continuous delivery, and big data. Also, we'll have some fun with stickies!

In this fourth episode, we share some impressions of Scrum Day Europe 2013 and Xebia's OpenSpace Knowledge Exchange. And of course, Serge Beaumont will have Fun With Stickies! First, we interview Frank Bakker and Evelien Roos at Scrum Day Europe 2013. Then, Adriaan de Jonge and Jeroen Leenarts talk about continuous delivery and iOS development at the OpenSpace XKE. And in between, Serge Beaumont has Fun With Stickies!

Frank Bakker and Evelien Roos give their first impressions of the Keynotes at Scrum Day Europe 2013. Yes, that was last year, I know. New, more current interviews are coming soon. In fact, this is the last episode in which I use interviews that were recorded last year.

In this episode's Fun With Stickies Serge Beaumont talks about hypothesis stories. Using those, ensures you keep your Agile really agile. A very relevant topic, in my opinion, and it jells nicely with my missing line of the Agile Manifesto: Experimentation over implementation!

Adriaan de Jonge explains how automation in general, and test automation in particular, is useful for continuous delivery. He warns we should focus on the process and customer interaction, not the tool(s). That's right before I can't help myself and ask him which tool to use.

Jeroen Leenarts talks about iOS development. Listening to the interview, which was recorded a year ago, it's amazing to realize that, with the exception of iOS8 having come out in the mean time, all of Jeroen's comments are as relevant today as they were last year. How's that for a world class developer!

Want to subscribe to the Xebia KnowledgeCast? Subscribe via iTunes, or use our direct rss feed.

Your feedback is appreciated. Please leave your comments in the shownotes. Better yet, use the Auphonic recording app to send in a voicemessage as an AIFF, WAV, or FLAC file so we can put you ON the show!

Credits

The Caffeinated Coder: Is Caffeine Good or Bad?

Making the Complex Simple - John Sonmez - Mon, 09/22/2014 - 15:00

For a long time I’ve wondered about the benefits or detriments of caffeine. I’ve always been one of those coffee drinkers who didn’t have to have coffee, but drank it when it was available. I’ve never really noticed how caffeine affected me, because I never really paid that much attention. But, I’ve always been curious, […]

The post The Caffeinated Coder: Is Caffeine Good or Bad? appeared first on Simple Programmer.

Categories: Programming

F#unctional Londoners 2014

Phil Trelford's Array - Sun, 09/21/2014 - 21:40

2014 has been another crazy year for the F#unctional Londoners meetup with over 20 sessions already. Thanks to our hosts Skills Matter we’ve been able to hold a meetup roughly once every 2 weeks.

Here’s a run down of the year so far and what’s coming up.

January

Ross kicked off the year with a deep dive to his LINQ enabled erasing SQL Type Provider.

Following on, in May, Ross left the sunny shores of Southend to tour the east coast with the talk covering NYC, Washington DC and Nashville along the way.

sql-provider

First seen at DunDDD in Dundee, Anthony’s excellent talk went on to be featured at CodeMesh London too.

With F# built-in to Xamarin Studio you can easily target iOS, Android and Mac.

February

Tomas returned to London to talk about his work on Deedle while at Blue Mountain Capital in New York.

As a follow on from the talk Tomas ran a hands on session using Deedle to explore world climate, the titanic, stock market trends and finally US debt.

March

There was a huge turnout for Scott’s hugely informative and at times somewhat amusing talk first seen at NDC London.

set phasers to null

Eirik Tsarpalis and Jan Dzik, from Nessos, presented their work on MBrace a programming model and cluster infrastructure for effectively defining and executing large scale computation in the cloud.

In this hands on treasure hunt session, Tomas presented a series of data extraction tasks using type providers to find words to build a sentence.

April

Rob Lyndon introduced Deep Belief Networks and his GPU based implementation in Vulpes. This talk was repeated last week at the prestigious Strangeloop conference in St Louis!

May

Michael travelled up from Brighton for a hands on session on building type providers. Type Providers are a hot topic in the London group with a number of popular type providers produced by members including FSharp.Data, SQLProvider and Azure Storage.

Mixing biology and physics to understand stem cells and cancer (video)

Ben Hall from Microsoft Research Cambridge gave a fascinating talk about his work with a hybrid simulator in F# to explore how stem cells grow (and some worms!).

Stephen Channell gave a repeat of his excellent talk featured at FP Days and the F# in Finance conference on liquidity risk.

Ian was in town to run a session at the Progressive .Net Tutorials and gave a repeat of his excellent talk from DDD North.

June

F#unctional Londoners regular Isaac, aka the Cockney Coder, talked about his professional work with Azure including his Azure Storage type provider.

In this hands on session we used the material from Mathias Brandewinder’s session in San Francisco to have some fun drawing fractal trees.

In this session Gabriele Cocco talked about his work on FSCL, an F# to OpenCL compiler.

July

Borrowing material from Mathias again, we built a 2048 bot using the open source web testing library Canopy.



Grant popped down from Leeds to run a fun code golf session where the aim was to complete a task with the least number of characters.

August

Phil Nash talked about how he was using F# scripting at work along side his some of his C++ projects.

In this hands on session we looked at the popular parser combinator library FParsec, building a mini-Logo parser and interpreter.

September

James popped down from Edinburgh to talk about his work with Philip Wadler on the open source project FSharp.Linq.ComposableQuery.

Goswin Rothenthal talked about his work using FSharp scripting in the design of the Abu Dhabi Louvre building:

Coming up this Wednesday we have Evelina talking about some of her data science work at Cambridge.

November

On November 6-7th the Progressive F# Tutorials make a return with expert speakers including Don Syme, Tomas Petricek, Mark Seemann, Andrea Magnorsky, Michael Newton, Jérémie Chassaing, Mathias Brandewinder, Scott Wlaschin and Robert Pickering.

ProgFSharp2014Don’t miss the special offer that runs up to the end of Evelina’s talk giving a 20% discount to members, brining the price down to a barmy 200GBP, use code F#UNCTIONAL-20.

Categories: Programming