Moments ago we introduced the Google Font API and Google Font Directory. In addition to Google’s support of web fonts, we’re excited to announce a second launch: a collaboration with Typekit to open source the WebFont Loader, a JavaScript library for improving the web font experience..

Google and Typekit believe that web fonts, in conjunction with the richer text styling offered by CSS3 and HTML5, provide the visual richness and high fidelity control of print typography while remaining accessible to machines and devices.

There are still some challenges when using web fonts, and the launch of an open source WebFont Loader addresses one of these difficulties: different browsers treat the download of web fonts differently. For example, Firefox will initially render a website’s text in the site’s fallback font until the web font is downloaded, and then the text will be re-rendered and re-flowed using the downloaded web font. Chrome and Safari won’t display the text until the web font is fully downloaded, and Internet Explorer sometimes won’t render any content at all until the web font is available.

The WebFont Loader puts the developer in control of how web fonts are handled by various browsers. The API fires JavaScript events at certain points, for example when the web font completes downloading. With these events, developers can control how web fonts behave on a site so that they download consistently across all browsers. In addition, developers can set the fallback font's size to more closely match the web font, so content doesn't reflow after loading.

Furthermore, the WebFont Loader is designed to make it easy to switch between different providers of web fonts, including Google, Typekit, and others. The code is modular, and we expect to add modules for other major web font providers in coming weeks.

Google is excited to work with Typekit to further web font technology. We look forward to working with others as well to continue to advance typography on the web.

By Jeremie Lenfant-Engelmann, Google Font API team

Today we announced Google App Engine for Business, with a host of new features to help enterprises run their business applications on Google’s infrastructure (read our blog post to learn more). We’re also excited to announce our work with VMware to connect our developer tools, making it possible to create rich, multi-device web applications that can be hosted in a variety of Java-compatible hosting environments. Call it cloud portability for the enterprise -- productively build apps that you can deploy onto Google App Engine for Business, a VMware environment (your vSphere infrastructure, your choice of vCloud partners, or VMforce), or other infrastructure such as Amazon EC2.

As part of this announcement, we’re providing early access to these tools -- you can start using them right now by downloading the latest milestone version of VMware’s SpringSource Tool Suite (STS). If you prefer to wait for the general release, you can sign up to be notified.

Spring Roo
With Spring Roo, a next-generation rapid application development tool, Java developers can easily build full applications in minutes, using the Java Persistence API (JPA) to connect to new or existing databases. Roo outputs standard Java code, so it’s easy to refine the back end with the SpringSource Tool Suite and the front end with the Google Web Toolkit SDK, using Roo as much or as little as desired.

Google Web Toolkit SDK
New data presentation widgets in Google Web Toolkit speed development of traditional enterprise applications, increase performance and interactivity for enterprise users, and make it much easier to create engaging mobile apps with a fraction of the investment previously required.

SpringSource Tool Suite
Using the Eclipse-based SpringSource Tool Suite, developers can now choose to deploy their application in their current VMware vSphere environment, in VMware vCloud, directly to Google App Engine for Business, or elsewhere. We call this cloud portability.

Google Web Toolkit Speed Tracer
Speed Tracer now helps developers identify and fix performance problems not only in the client and network portions of their apps, but also on the server. By incorporating server-side time traces from both Spring Insight and Google App Engine AppStats, Speed Tracer provides a consolidated view of where sluggishness actually comes from -- be it client, network, or server -- so it’s much easier to see what to fix.

If you’re building business apps, we hope you’ll find these tools make it easier and more fun to get your job done. Maybe you’ll save time and money by developing and testing apps on App Engine, and then deploying to your VMware environment. Maybe you’ll run a second instance of your apps on App Engine for disaster recovery. Or, maybe you’ll take your existing on-premise apps and extend them to the web and to mobile devices in a fraction of the time it might have otherwise taken. The point is, whatever you decide, you can be confident that it’s possible, and that you’re not locked in.

We’re happy to be working with VMware to bring increased interoperability, portability and choice to enterprise developers. We invite you to download the latest milestone version of STS and try these tools for yourself.

By Bruce Johnson, Google Developer team

We launched Google App Engine two years ago to enable application developers to rapidly build and scale their apps on Google’s infrastructure, without having to worry about maintaining their own servers. Today, we’re excited to bring this platform to IT departments, with the announcement of Google App Engine for Business. Google App Engine for Business lets organizations build and maintain their applications on the same scalable architecture that powers Google applications, with added management and support features tailored specifically for the enterprise.

Google App Engine for Business introduces a number of new features that our enterprise customers have been asking for, including:

• Centralized administration: A new, company-focused administration console lets you manage all the applications in your domain.
• Reliability and support: 99.9% uptime service level agreement, with premium developer support available.
• Secure by default: Only users from your Google Apps domain can access applications and your security policies are enforced on every app.
• Pricing that makes sense: Each application costs just $8 per user, per month up to a maximum of $1000 a month. Pay only for what you use.
• Enterprise features: Coming later this year, hosted SQL databases, SSL on your company’s domain for secure communications, and access to advanced Google services.

With these new features, we’re making it easier for businesses to take advantage of the core benefits of Google App Engine: easy development using languages you already know (Java and Python); simple administration, with no need to worry about hardware, patches or backups; and effortless scalability, automatically getting the capacity you need when you need it.

Google App Engine for Business is currently in preview, opened to a limited number of enterprises. Learn more about how you can participate, and check our roadmap to follow features as they become available.

By Sean Lynch, Google App Engine team

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Roundarch's Technical Architect Greg Knapowski and Sr. Front End Developer Lawrence O’Sullivan who will be demoing as part of the Developer Sandbox.

Our client, NYSTROM Herff Jones Education Division, the leading producer of maps and globes for schools, sees many classrooms replacing chalk boards and wall maps with interactive white boards and laptop computers. Our Roundarch team worked to develop StrataLogica™, a web-based learning application that delivers map and atlas content in an engaging 3D, interactive environment utilizing the Google Earth API.

The Google Earth API was used as a foundation for StrataLogica™ to make use of its sophisticated image rendering logic, satellite imagery and access to built-in tools and navigation controls. As an enterprise scale application, we faced some interesting challenges and gained many insights along the way that we’d like to share.

Our first task was to prove we could wrap Nystrom’s existing educationally-focused maps and globes onto Google Earth while retaining the same high quality resolution delivered in their print products.

Achieving acceptable image resolution resulted in file sizes which were much too large. In addition, we needed to deliver an increased level of map content and granularity of images as the user zoomed into the earth. To address these two issues, we created a custom process that takes an Adobe Illustrator file and outputs Superoverlays in accordance with KML 2.1 standards. Using open source Python frameworks, we created a customized solution that outputs Superoverlays with various levels of content.

Our next challenge was to provide support for authoring and maintaining content, in the browser, using the Google Earth plugin. All content is authored and maintained in a content management system (CMS) in much the same way as any dynamic website. One unique difference is that some of the content elements are geo-referenced coordinates that specify the location of content on earth. In the case of placemark balloons, the geo-referenced coordinates identify “hotspots” on the Nystrom maps which become clickable when the user turns on a setting. The placemark balloons provide supplementary audio, image, video and descriptive content such as the example shown above for the Appalachian Mountains.



An ETL (extract, transform, load) process is used to transfer published content into the StrataLogica™ application database. We then use Java Server Pages (JSPs) to generate the KML which renders the placemark balloons. The KML contains both the “hotspot” coordinates and embedded HTML to display the audio, video, images and text. This example illustrates that standard web technologies can be used to deliver content through the Google Earth API with the exception that KML is generated in addition to HTML.

Once Nystrom was able to see their maps and content rendered on Google Earth through the plug-in, we needed to create a compelling and immersive educational experience to allow teachers and students to interact with maps, not just view them. Our team added features such as marking tools, custom content authoring tools and the ability for users to share and collaborate. For those of you attending Google I/O, stop by our booth in the Sandbox space to experience StrataLogica™ firsthand and chat with our team about the front-end development challenges we faced and how we overcame them. If you aren’t attending we encourage you to check out our blog post at http://blog.roundarch.com where we share more about how we developed StrataLogica™.

Posted by Greg Knapowski, Technical Architect, and Lawrence O’Sullivan, Sr. Front End Developer, Roundarch

When you start reading this book, you will quickly understand that the authors are affiliated with IBM. This is nothing wrong per se, but this seems to influence too much the vision that the book proposes, ignoring approaches proposed by others. Including “iterative” in the title seems here to be only a marketing trick used to make it catchy. They don’t give you a precise definition of “iterative”, saying rather than it is a “modern method” (Tom Gilb was talking about evolutionary development 30 years ago) and that iterative management is result-based rather than activity-based. The difference between iteration and increment is not discussed. The IBM bias is visible when they state for instance that RUP is a “well accepted benchmark of modern iterative process”.

The beginning of the book describes the development process, providing a two pages description of COCOMO in the middle of it, and proposes way to improve it. The solutions focus mainly on an initial emphasis on architecture and modeling (naturally it is UML with some tools) associated with the recommendation for code reuse or component integration. I was hoping to find more useful material in the part dealing with “practical measurement for software engineering”. However the material is arranged around RUP phases and I found only one metric formula in the 50 pages devoted to this topic. There is some interesting content in this part, but it is only very high level.

This short book is close to be only a white paper for the IBM Rational Unified Process approach. It contains some interesting material for a software development manager wanting to think about introducing metrics, but its biased IBM approach and lack of directly useful content make it awkward for the experienced development professionals.

Reference

“The Economics of Iterative Software Development”, Walker Royce, Kurt Bittner and Mike Perrow, Addison-Wesley, 171 pages, IBSN 978-0-321-50935-2

Get more details on this book or buy it on amazon.com
Get more details on this book or buy it on amazon.co.uk

Quotes

The most significant way to improve economic results is usually to achieve a software solution with the minimum amount of human-generated material. Our experience shows that managing scope and raising the level of abstraction through component-based technology and service-oriented architectures are the highest leverage techniques that make a difference

External stakeholders, including customers and users, cannot expect initial deliveries to perform up to specifications, to be complete, to be fully reliable, or to have end-target levels of quality or performance.

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Adam Kalsey, Developer Evangelist at Voxeo Developer Network, who will be demoing as part of the Developer Sandbox.

Connecting to your customers via voice, SMS, or even IM allows you to reach them when they aren't at their desks. These real-time communications mediums are skyrocketing in usage and ubiquity, as what is more ubiquitous than the phone? Building an application that works over the phone or instant messaging is easier than you think.

Telephony has traditionally required complicated hardware and software or specialized programming languages. In recent years, however, web programming tools and models ranging from VoiceXML to web services APIs have made it easy for any developer to create communications services.

Companies for years have been providing customer self-service applications as Interactive Voice Response (IVR) systems as a way of boosting customer satisfaction while cutting service costs. These techniques and tools are starting to make their way into other mediums and at the same time are coming down in price, making them accessible to a whole new class of developers. This is leading to some great innovation as developers find creative ways to weave phones and text into their applications.

It's not just voice. A rising percentage of the population is just as likely to communicate over text messaging, IM, and social networks as they are to pick up the phone. APIs and customer-self service systems must adapt to the preferences of the customers they want to help. Over the next decade, 50% of the automated communications that today happens over the telephone will shift to other mediums. It's important for the applications you develop to serve your customers to
be available on the networks they choose to use.

Here's how an IVR application might work over instant messaging. Your company provides a Jabber IM address for people to contact you. Any of your customers using Google Talk can strike up a conversation. Your code intercepts their message and routes it to the right department based on the keywords used in the chat. Perhaps you even respond off hours letting people know when your agents are available or answer common questions automatically from a knowledge base.

Have you ever wanted your Google Talk number to be answered by your code? When your friends call, you'd like to pick it up, and when telemarketers call, you’d like it to go straight to voicemail. In addition to accepting calls, your application can make calls, sending appointment reminders or triggering alerts using only a few lines of code. You can even set the caller ID of the outbound call to your Google Voice number so that return calls come to you.

To demonstrate how easy it is for a developer to weave voice and text into their application, lets look at a sample application running on our Tropo cloud communications platform.

We've created a mashup for the San Francisco Bay Area Rapid Transit (BART) schedules so you can see when the next train is due to arrive at the station of your choice. Call or SMS (407) 374-9954 or send an IM to bartdemo@tropo.im with Google Talk to try it. You can even talk to @tropobart on Twitter. Tropo treats Twitter as an IM network, sending all @mentions to your application just like an instant message would be.

Tropo works by converting phone calls, IMs, and text messages to API calls that your code works with. An incoming call or message is delivered to your code and your code tells Tropo what to say back.



The code behind this can sit in our cloud or on the web server of your choice, including Google App Engine. Other sample applications are available covering everything from simple games to directory assistance, to checking into Foursquare.

Real-time communications APIs for voice allow you to give instructions to the phone system and IM networks to answer or make calls, accept text messages, interact with your users using speech synthesis and speech recognition, set up conferences, or just about anything else you can think of and mash it up with Google technologies, like this Wave robot that adds conference calls to any Wave.

To learn more about building voice and text applications and the real-time communications APIs that are available to you, visit Voxeo in the Google I/O Developer Sandbox or visit the Voxeo blog and Tropo blog.

Posted by Adam Kalsey, Developer Evangelist at Voxeo Developer Network

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Jeremy Glassenberg from Box.net, who will be demoing as part of the Developer Sandbox.

Google's rich platform for developers is matched only by its variety of innovative web services. So when working with Google APIs, why use only one or two Google services?

When launching with the Google Apps Marketplace in March, Box.net could see opportunities in connecting with Google services across the board, from sharing files in Gmail to enhancing collaboration in Google Sites. But how could we connect to half a dozen different services, while making the interaction with a partner completely seamless and easy to use?

Luckily, for those who want to go all-out on the Google platform, Google's diverse APIs provide what you need to code efficiently, and organized to connect everything in a way that makes sense. Here is what we’ve found helpful to help you optimize your development experience:

1. Standards across APIs

By leveraging Google's support for the OpenID and Oauth protocols, we can not only connect Google users to access Box.net easily, but also to make all features of Google accessible from Box. This way, when Google Apps accounts connect to Box, we can automatically connect users to Google Docs, Google Sites, Google Calendar and Gmail through the magic of a secure, centralized single sign-on process.

2. APIs and Gadgets make for a seamless integration

We wanted to make it easy for users to access services like Gmail and Google Docs from within Box. With our OpenBox Developer Platform, we could make those Google services accessible through our simple file menu. Users could right-click on a file, and choose to "Share in Gmail", "Edit with Google Docs", etc. But we wanted to go even further, and integrate "inside and out" - making Google accessible in Box, and Box accessible in Google's services. This was made possible with Google Gadgets.

By adjusting gadgets we made for OpenSocial partners such as LinkedIn, we could easily create Box.net gadgets that also work with iGoogle, Google Sites, and even Gmail Labs and Google Calendar Labs. These "Box.net inside Google" gadgets became surprisingly popular among new users.

We even found it possible to connect Google services together more deeply through these two-way integrations. By making our OpenBox features available in our Google Gadgets, our users could edit a document from Box in Google Docs, or share a file into Google Sites, from within the Gmail sidebar.



3. Focus on user experience

With multiple connections to Google services in Box, and the placement of Box.net inside four Google services via Google Gadgets, this became an exciting engineering project. But we couldn't forget that everything needed to work in a straightforward, easy-to-use manner for businesses in Google Apps. Luckily, Google Apps' design and installation process for the Marketplace helps to bring just about everything together for the user.

The installation process enables you to obtain the OpenID information for a Google Apps administrator, access rights to various Google APIs, and connect users in the same Google Apps domains automatically. This sounds great for your code, but pay attention to the user experience. Google Apps administrators need to understand what they're about to make accessible to their users. Sub-users of the Apps account need to be able to sign up easily as well.

Users and administrators will also want to choose which services to use. In our case, some users wanted to edit Google Docs in Box.net, while others just want to access Box.net from within Gmail or Google Sites. Thus, just like within Google Apps' administrative console, Box.net made individual components easy to enable and disable. We also provided detailed instructions for adding and using each piece of the integration.

---

At Box, we look forward to continuing work with Google, integrating even more deeply as Box.net and Google both expand our platforms.

Posted by Jeremy Glassenberg, Platform Manage at Box.net

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Jim McNelis, co-founder and principal of Dito, who will be demoing as part of the Developer Sandbox.

As an early Google Apps Authorized Resellers, Dito, an IT services company I co-founded in 2007, has become a trusted advisor to a diverse group of organizations using Google Apps. We’ve established strong relationships with each of our customers and are a go-to source for information on specific features and functionality, best practices, and work flow processes. This gives us a unique insight into end-users’ needs as they ramp up on Google Apps, and we continually work to improve our service offerings and our customers’ experiences.

Over time, we compiled feedback from our clients and noticed an interesting trend - customers wanted a better way to manage their shared contacts within Google Apps. Out of the box, Google Apps handles shared contact management via API’s, as opposed to a Graphical User Interface (GUI) which is most common. Many small organizations don’t have the resources to develop a solution to interface with API’s. So, we decided to develop one for them that could easily integrate with Google Apps to allow administrators to manage, and end-users to browse and search, shared contacts within their domain.

We appreciate the reliability and scalability of cloud-based solutions and wanted our application to live “in the cloud” as well. After extensive research, we determined that Google App Engine was the right platform for our application since it works seamlessly with other products and tools within the Google ecosystem like Google Web Toolkit and the Google Apps APIs. From there, we set out to develop our app: Dito Directory for Google Apps.

Dito Directory represented Dito’s first foray into the development business. Sure, our developers had years of experience, but our team was new and we were preparing to develop apps on the bleeding edge of technology. In years prior, developing an application with this functionality and accessibility would have been an expensive and exhaustive process which would have required client-side application development and everything that goes with it.

Google App Engine and the Google Apps Marketplace erased these challenges and provide us with a simple yet elegant platform and distribution channel to deliver our application to the masses. From concept to delivery, the application took two developers less than four months to complete - just in time for the launch of the Google Apps Marketplace.

Since then, almost 1000 Google Apps admins have installed Dito Directory on their domain resulting in numerous happy admins and end-users. I hate to say it, but it wasn’t hard to get 1000 people to install our app either. We basically relied on our existing customer base and the popularity of the Google Apps Marketplace to advertise our apps. There has been not been a significant marketing push or advertising dollar spent...we simply developed the application, launched it on the marketplace, and now watch about 15 domains install Dito Directory every day.

With all of the installs, we received great feedback from our customers, and our team continues to improve the functionality and performance of the app. As a result, Directory continues to evolve to meet customers needs. We don’t have to guess what users want, we simply listen to what they have to say and iterate accordingly. And since our app is cloud-based, upgrades are easy. There is no additional software to install. After we release new features, simply refresh your browser and the app is updated.

During Google I/O this week, our team will showcase Dito Directory in the Enterprise pod at the Developer Sandbox. If you’re attending the conference, please stop by to chat about our app, give us feedback, or just say hello. We’ll also be giving away the paid version of our app to 10 lucky winners at the conference, and other exciting prizes. I hope to see you there.

Posted by Jim McNelis, co-founder and principal of Dito

Collaboration: The New Duct Tape
Thomas M. Cagley Jr.

The economic crisis of the past few years has caused organizations great and small to actively chase innovation.  Just what exactly innovation is and how it can be measured is are hotly debated topics.  Even though everyone wants to be an innovator, we will ignore the definition of innovation for the time being and focus on one of the primary tools being leverage in the pursuit of innovation, collaboration.  We will focus on collaboration because done correctly; collaboration is the duct tape of the business world. 

Collaboration is typically defined as two or more people working together to solve a common problem or in pursuit of a common goal.  Two components of the definition get to the crux of what makes collaboration work.  The first is the phrase “two or more people” because you can’t collaborate by yourself.  Without different points of view much of the power of the technique is lost.  Second is the phrase, “common problem” or “common goal”.  The common goal acts as the focusing mechanism for the overall process.  Combining these two concepts means that collaboration works through social interactions focused on a common mission.  Done correctly collaborative efforts deliver synergy, addresses complexity and fosters involvement.

The makeup of the team is a major component of the success of any collaboration.  Rogers and Hammerstein combined scores and lyrics into fantastic songs, creating something greater than the simple sum of the parts.  Great sports teams attain championships when they combine complementary skills which allow them to tackle problems that they are not capable of addressing individually.  In brainstorming we are taught to build on the ideas of others; to dovetail ideas.  Combining and building on the ideas and thoughts of the team is synergy.

Businesses don’t create collaborative teams to tackle the simple problems.  When was the last time you formed a collaborative team to decide when you were going for lunch?  Never?  But there probably was a group created (hopefully collaborative) to decide where the entire office would go for the last holiday lunch.  Collaboration is a reflection of the maxim that two heads are better than one (on different people).  Many times multiple points of view are required to solve problems that can’t be easily described or solved individually.  Collaborative groups allow problems to be viewed from multiple points of view which allows the team to describe problems and solutions in a more robust manner which is one technique for dealing with complexity.

Collaboration can used to build buy in for the solutions that are generated.  Naomi Karten in her book, Changing How You Manage and Communicate Change, said “that even a minimal sense of control can go a long way towards easing the stress that people feel.”  Collaboration is one means of providing a means for participation yielding a sense of control.  Each voice on the team is a representative of a constituency that is being voiced as the solution is crafted.  As part of implementing the solution it must be expected that the individual representatives support and sell the solution to their constituencies. 

Part 2 –Attributes of Collaborative Teams

I'm posting this from Moscone West, the site of Google I/O 2010. Some things that it may be useful to know:

• The official hash tag is #io2010

• The keynotes will be live-blogged all over the place (thus, not here) and also live-streamed on YouTube.

• If you're mad because you couldn't get a ticket, we're sorry, but you're not alone. We have filled Moscone West to the very absolute splitting-at-the-seams max. There are a ton of Googlers who couldn't get in.

Things have already started; above is our own Roman Nurik giving the first Android session at the pre-IO Bootcamp.

Above are the signs over the Android part of the main gathering area. There are many tantalizing display cases and tabletops; gleaming and empty now, but not for long.

Meta-Blogging

Thursday is the big Android day at Google I/O. There will be announcements. We're going to try to use this blog to deliver more depth and perspective than you can expect from an executive on a stage in a keynote. So we've lined up a whole lot of blog posts; more than any human could be expected to read in a day.

The plan is to have a couple of posts on Thursday summarizing and highlighting what we think are probably the big-news stories. Then we'll have more detailed technical pieces every other day or so for the next couple of weeks, filling in the details behind the announcements. It's heart-warming how eager the engineers are to tell the world about what they've been building, and it makes me happy to facilitate the process.

There are other Google blogs that are going to be working hard too, telling the other stories coming out of this gathering. We'll post some links to things that seem particularly newsworthy, either to the world in general or to the Android comunity in particular.

Stay tuned!

I have been experimenting with standups a lot over the last 4 years and I found a good way to get the most from it. In my experience a good standup only serves one goal. It is to unite every mind in the Team into One Mind so there can be one view on [...]

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Ryan Massie, Product Manager at Clicker.com who will be demoing as part of the Developer Sandbox.

Over the years, the technical gap between engineering for the Web and engineering for an operating system has continued to narrow. Third party plug-ins led the way in this progression, until their services eventually became standardized and built into the browser itself. Now, using HTML5, we're outgrowing many of the limitations that came with developing for a specific OS or device, enabling us to create next generation user-experiences that bring more utility to more platforms than ever before.

In general, building for the Web enables us to be more agile. By establishing implementation guidelines for video, animation, audio and storage, HTML5 further reduces the level of fragmentation across the Web, empowering developers to efficiently build rich, browser-based experiences of superior quality.

For us here at Clicker, that means opportunities to create more unique and immersive experiences that guide users to the premium-quality TV shows, movies, music videos and Web originals available to watch online. Using HTML5, we’re able to offer more innovative in-browser experiences with movement and reflections while providing a faster performance.

One reason the Web is an amazing platform is because changes made to a given website are instantaneous and can be viewed by anyone in the world with an Internet connection. By creating browser-based utilities, optimized for various screens and devices, we can continue to create consistent, reliable and scalable experiences that push the boundaries of what people can do online, making the Web an increasingly more interesting place. And as developers, it's important to make the Open Web a success. HTML5 and other advancements give us more opportunities to stretch the boundaries of what we can enable people to do. And, without the hindrance of downloads and the added benefit of increased browser speed, consumers will increasingly expect rich, browser-based experiences. This end-user demand will continue to spur the overall pace of innovation and development on the Open Web.

We look forward to meeting fellow developers and continuing the conversation in person at I/O this week!

Posted by Ryan Massie, Product Management, Clicker.com

Back in February I started work on integrating Yahoo’s FireEagle location service into Geomium and I ran into a problems with Python 2.5. Using Steve Marshall’s Python library the included test.py script was working perfectly with Python2.6, but when running with Python2.5 I’d get back an “Invalid OAuth signature error”.

I posted the problem to the OAuth user group but didn’t get any response. I got in touch with Yahoo. After quite a bit of back and forth we finally figured out the problem, which I’m posting here to try and save others from months of frustration!

The Yahoo guys noticed that with Python2.5 the HTTP host header was being sent through as as “fireeagle.yahooapis.com:443″, whereas 2.6 sends “fireeagle.yahooapis.com”. The inclusion of the port results in an invalid OAuth signature, because the signature is generated assuming the port isn’t included. I dug into the Python2.5 httplib code and came across this:

 813    if self.port == HTTP_PORT:
 814        self.putheader('Host', host_enc)
 815    else:
 816        self.putheader('Host', "%s:%s" % (host_enc, self.port))

In Python 2.6 the comparison on line 813 is done with self.default_port instead of HTTP_PORT, which prevents the port from being added with HTTPS requests. I noticed that later on in the code that if you pass in your own host header it prevents one being created for you:

 875     def _send_request(self, method, url, body, headers):
 876         # honour explicitly requested Host: and Accept-Encoding headers
 877         header_names = dict.fromkeys([k.lower() for k in headers])
 878         skips = {}
 879         if 'host' in header_names:
 880             skips['skip_host'] = 1

So the fix turns out to be really simple – explicitly set the http header. That’s exactly what I’ve done in my fork of the fireeagle library (see the fix). I’ve also sent a push request, so hopefully this fix will make it back into the original library. Thanks to Arnab Nandi and Anand S from Yahoo for helping to debug things their end.

Unicorn's been a topic I've been interested in learning about for a while now; numerous Engine Yard customers and developer friends use it, love it, and recommend it. Thankfully, the opportunity to do so recently presented itself. I spent some time poking around free resources looking for answers to my questions, and it wasn't as easy as I'd hoped... so I decided to go straight to the source.

First, I spent a bunch of time going over the Unicorn README file. While comprehensive, when I was done, I still had questions, so I put them all together, and emailed the Unicorn development team. They were gracious enough to reply with detailed answers to all my questions, and now that I'm in the know, I figured this would be a great resource to share with the rest of you. It's not our usual style of blog post, but it's solid information just the same, in what's hopefully an easily consumable format.

I've organized the questions into topical sections. The topics are: Clients, Debugging, Process Management, Load Balancing, Thread-safety, Rack support and Rack wrapper, Log Files, Binary Upgrades, Forking, Listening Interfaces, Configuration, Asynchronous Transfers, The Binary and Dependencies. There's a lot, so read through it all, or skip straight to the section that interests you.

Clients What are "fast-clients"?

Clients that can make full (or close to full) use of the network bandwidth available to the server. Clients on a LAN (or the same host) usually fit this description, as they don't have to trickle data to the server over a slow link.

What's a slow client, by comparison?

A client with high latency or limited bandwidth that forces the server to sit idle and wait for data in the request or writable buffer space in the response. Accept filters in FreeBSD and deferred accept in Linux mitigate this problem for slow legitimate clients, but a dedicated attack can still get around those.

Slowloris is perhaps the most prominent example of the damage that could be caused by slow clients, but there have been similar tools like it floating around privately for years, including the Unicorn author's own "David" tool, which he (David) only made public after Slowloris:

Clients that sit around with idle keepalive connections is also huge problem for simple servers like Unicorn (and traditional Apache prefork), so Unicorn does not support keepalive.

The Unicorn author also works on the Rainbows! server, which is designed specifically to handle talking directly to slow clients and high-latency apps (Comet/WebSockets) without nginx in front.

What is a "low-latency, high-bandwidth connection"?

Anything on localhost or the local area network that doesn't make the server sit idle, unable to service other requests.

Debugging Can you give me an example of how to debug?

Reproducibility is critical to debugging. Processes are inherently simpler, as the process state is always well-defined on a per-request basis and isolated from other requests as much as possible by the OS. One example is to help track down a memory leak related to a specific class of requests:

An non-Rubyist admin noticed that among a pool of workers, some used significantly more memory than other workers.  Since the log file format always logged the PID serving each request, they were able to quickly narrow down which endpoints were prone to leaking memory (without even looking at the code).

In a server where requests are all served within the same process, it would've been much harder to narrow down which endpoints were using up memory. In a server where a single process handles multiple clients simultaneously, it would've required thorough inspection of the source code to track down which requests were leaking memory.

Process Management

"Unicorn will reap and restart workers that die from broken apps. There's no need to manage multiple processes or ports yourself. Unicorn can spawn and manage any number of worker processes you choose to scale to your backend."

Does that mean that Unicorn doesn't need monit or god?

No server needs things like monit or god; it all depends on your comfort level, your app, and your support requirements. It's always possible—albeit unlikely—for the master process to die, but things like monit and god aren't immune to dying, either. Developers use those tools, and similar ones, like Bluepill, with Unicorn.

Load Balancing

"Load balancing is done entirely by the operating system kernel. Requests never pile up behind a busy worker process."

So there isn't a mongrel queue issue?

No, there no a Mongrel queue issue on a single machine.  A single queue is shared across worker processes and the workers only pull off the queue when they're available to do work. There's still a potential queue issue in a cluster behind a load balancer, but the risk is mitigated, since most servers are multicore and run multiple worker processes. The queue is also tunable by specifying the :listen parameter.

Thread-safety Why is thread-safety good?

The utility of thread-safety really depends on the particulars of your situation. It gives you much more flexibility with what your app can run, and under ideal conditions, threads are memory efficient and relatively inexpensive. Thus, allowing apps to work with threads is good for experienced programmers.

On the other hand though, making things thread-safe by default can hurt performance in single-threaded situations. Even contention-free locks can end up adding significant overhead due to memory barriers. Both MRI and Python core developers have come to this same conclusion.

Rack Support and Rack Wrapper What rack applications are supported?

Pretty much anything that passes Rack::Lint (and sometimes, even a few that don't).

What Ruby on Rails versions does the wrapper support?

The manpage says everything 1.2.x to 2.3.x, and there are integration tests for those version.

Log Files

"Builtin reopening of all log files in your application via USR1 signal. This allows logrotate to rotate files atomically and quickly via rename instead of the race condition prone and slow copytruncate method."

What is the USR1 signal?

USR1 is the first user-defined signal, which usually gives applications the most flexibility in determining what a signal handler for it would do. To send a USR1 signal to Unicorn, use the standard kill(1) command:

kill -USR1 $PROCESS_ID

Nginx also uses the USR1 signal for reopening log files.  Most of the signals Unicorn accepts map directly to the nginx ones for ease-of-learning. Unicorn also takes steps to ensure multi-line log entries from one request all stay within the same file.

Binary Upgrades What are binary upgrades?

Binary upgrades are upgrades that upgrade Unicorn itself, the version of Ruby, or even any system libraries including the system C library. For users that depend on copy-on-write functionality, it's also the only way to upgrade the application

How do you upgrade?

The upgrade procedure is the same as nginx, and is also documented here (bottom of page).

What happens after upgrading?

After ensuring the old processes are terminated gracefully (via SIGQUIT), that same code should ensure that the app behaves as expected. If the app is broken, another "upgrade" is required which may involve switching back to a known good version.

Forking What is the preload_app directive?

The preload_app directive loads the application before forking workers, so it can share any loaded data structures. By default, workers each load a private copy of their app for out-of-the-box compatibility with existing servers.

What's a use case for using the preload_app directive?

preload_app can dramatically speed up startup times. It can also make it easy to share memory across processes when using Ruby Enterprise Edition. REE also uses tcmalloc on some platforms, like Linux, instead of a generic malloc, which improves performance for most server workloads independently of copy-on-write.

Listening Interfaces What's an example configuration for how to set this up and how this can be used for debugging an application?

You can set up a worker to listen on a specific address so that you can do things like strace the worker while hitting that address and see what happens. There's a commented out example in here, which is shortened and uncommented here:

after_fork do |server, worker|
  # per-process listener ports for debugging/admin/migrations
  addr = "127.0.0.1:#{9293 + worker.nr}"
  server.listen(addr, :tries => -1, :delay => 5)
end

Normally, strace will slow down a process enough that it usually "loses" when trying to accept() a connection against other workers and it never sees the request.

Configuration Is there a good example configuration to help me get started?

The examples here cover many settings, including comments. The simplest case is with preload_app=false. Here's a short example:

worker_processes 16
pid "/path/to/app/shared/pids/unicorn.pid"
stderr_path "/path/to/app/shared/log/unicorn.stderr.log"
stdout_path "/path/to/app/shared/log/unicorn.stdout.log"

In contrast, preload_app=true can significantly complicate things, as it requires disconnecting/reconnecting to the database and other connections to avoid unintended resource sharing. All configuration settings are documented in the RDoc of the Unicorn::Configurator class.

In addition to Unicorn::Configurator settings, there's also the rackup config file (usually config.ru) used by all Rack applications independently of the underlying server. There's also system/kernel tuning, which the Unicorn documentation touches on here.

The Binary What is the unicorn executable?  What is the unicorn_rails executable?

The unicorn executable is a Rack-only tool modeled after Rack's "rackup" and is recommended for Rack applications. unicorn_rails was made to be an easier transition for users of pre-Rack versions of Rails. The manpage encourages Rails 3 users to use plain unicorn instead.

What's the difference?

From the unicorn_rails manpage, some conventions of unicorn_rails are modeled after script/server found in Rails. It creates directories under "tmp" like script/server and the -E/--environment switch sets RAILS_ENV instead of RACK_ENV.

Dependencies Are there any dependencies?  Gems, system packages, etc.

Rack is the only Gem Unicorn currently depends on.  Unicorn does not set hard dependencies on any released version of Rack. Unicorn depends on MRI 1.8 or 1.9 on a Unix-like platform.  There have been commits to make the C/Ragel HTTP parser work with Rubinius, but there have been some other issues in the pure Ruby code. Building from git requires Ragel (but the distributed source tarball/gems do not). The project does not distribute precompiled binaries.

Unicorn uses RDoc for most of the documentation and John MacFarlane's Pandoc (a Haskell tool) for the Markdown manpages. Pandoc was the most prominent Markdown to manpage converter at the time, as Ryan Tomayko's ronn had not appeared when the manpages did.

What are the requirements?  Operating system, ram, etc.

Most POSIX-like platforms are supported. Unicorn depends on a bunch of Unix-y things like fork(), the ability to share file descriptors with children, signals, pipes, unlinked open files, etc...

Unicorn has been deployed to and tested on various Linux distros heavily. The Unicorn mailing list has gotten reports and patches for OpenBSD compatibility, too, so that should work. Unicorn does not depend on any exotic system calls not provided natively by MRI.

RAM usage depends heavily on the application/libraries, version of Ruby, word size of the architecture, and number of worker processes configured.  It shouldn't take significantly more or less than any other Ruby web server.

Conclusion

I didn't start out with a specific problem, more like a void in my knowledge base, and now that that void is gone, I'm pretty pleased with the robust capabilities of Unicorn. It's not going to be the tool-of-choice for every use case, but clearly, it'll do wonders in a lot of them. As always, leave questions and comments here!

Following the sneak preview of Gmail contextual gadgets during the launch of the Google Apps Marketplace, we announced a limited trusted tester period for those who couldn’t wait to get started. We were pleased with the overwhelming interest in the API, and were able to invite a select group of developers to participate.

We are happy to announce that the Gmail contextual gadgets API is available to all Google Apps Marketplace developers today. For more on how to get started, check out the documentation and read the full announcement on the Google Enterprise Blog.

For anyone interested in a deep-dive on Gmail contextual gadgets, make sure to check out my session at Google I/O!

Posted by Dan Holevoet, Gmail contextual gadgets team

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by myERP.com’s Francois Nadal (CEO) and Thomas Ricard (CTO) who will be demoing as part of the Developer Sandbox.

myERP.com is a full ERP solution that has been available in the Google Apps Marketplace since its launch in early March.

Our all-in-one business suite includes all the functions a company needs to run their business from CRM to accounting to supply chain management. We've gotten a tremendous response from customers: in less than 3 weeks, we signed up more than 3,000 new small and medium customers.

Getting started was not easy. First, we had to convert our Java application to the cloud, using Google Web Toolkit (GWT). It was a task that took us nearly 2 years to complete — we launched the first released of our ERP in the cloud in late 2008. It attracted the interest of Google engineers who subsequently invited us to present at the Google I/O 2009 developer conference in San Francisco.

To prepare for the Google Apps Marketplace launch, we started modifying our codebase. There were two very separate tasks to achieve. The first one was to OpenID-enable myERP.com, and more specifically SSO-enable it. A Google Apps user should never be asked to enter his or her credentials to login to myERP.com. When you work with Google, you have to focus even more than you are used to on the user experience. The user should never see the underlying technology and just navigate from his Gmail inbox or his Google Calendar to myERP.com as if everything was a single application. That was the toughest part to do, since our application was not ready for it. So, we did a lot of refactoring. We're proud today to use our myERP.com software through our own Google Apps account.

The second part was easier since we were already integrated with Google Contacts and Calendar long before the Google Apps Marketplace. The main change was that we used to ask individual users for their credentials, which was a security risk. Now, we use OAuth. This development took us 2 weeks. We currently use the Google Contacts API to retrieve (import), create, update and delete customers created in myERP.com. We also use the Provisioning API to synchronize accounts in myERP.com with accounts in Google Apps. MyERP.com now lets customers use Gmail and Calendar to manage their Business Actions. Our next Google Apps development will focus on Google Docs: frequently requested features include exporting data to a Google Spreadsheet and putting an invoice in a Google Document.

We will be demoing in the Developer Sandbox at Google I/O, and we look forward to seeing you there!

Posted by Francois Nadal (CEO) and Thomas Ricard (CTO), myERP.com

This post is part of the Who's @ Google I/O, a series of blog posts that give a closer look at developers who'll be speaking or demoing at Google I/O. This guest post is written by Amit Kulkarni, Co-Founder and CEO of Manymoon, who will be demoing as part of the Developer Sandbox.

Manymoon is a free social productivity tool that helps teams manage and share projects, tasks and conversations. We became part of the Google Apps Marketplace at launch and have seen tremendous early success. Over 1,000 new businesses sign up each week, making us one of the top apps in the Marketplace. Below are some of our tips for building a successful app:

1. Define the problem - and make sure it's a big one!
For us, that was all the communication that gets lost in email. We thought email is great for notifications and is great ... as of 10 years ago.
But the internet is now social and about communicating for fun and work and both! And Google Apps has 25 million users and is signing up over 3000 new businesses a day.
We realized there was a significant opportunity.

2. Simplicity Sticks
Building a web app is different than building an enterprise app. The premium is on intuitiveness and ease of use. You've lost if the user has to read a manual, FAQ or go through a tutorial. User attention spans are short and it's critical for them to get up to speed in minutes. And that meant starting with something very simple: sharing tasks and projects.

3. Focus on What You're Great At
While in private beta, we had numerous customers asking for the ability to add documents to tasks and projects. The more we discussed with customers the more we realized that they were asking for features outside of our core vision. For example, they were asking for things like version control and co-browsing. This would have taken us months to build and is a whole different startup in fact! Instead, we decided to integrate with the Google Docs APIs. From a technical perspective, it provided us with the features our private beta users needed and it only took us a few weeks to complete! We immediately saw significant traction with users since our daily visits increased by 300%.

As we receive feature requests from users, we always look to see how we can integrate existing Google Apps to provide a better user experience. Another great example is our calendar feature. Customers were asking for a graphical calendar feature that included all necessary project information: open tasks, events and milestones. We were able to use the Google Calendar API and in just a few days provide this feature to users. We did this by using the API to automate multiple steps: create a calendar, share it with people and seamlessly update it with the latest project information.

4. Re-Use and Drive Engagement
One major benefit of using Google APIs was the way it reduced the friction to get users engaged. We did this by re-using each user's existing data in Manymoon! Specifically, a user can use Manymoon for tasks and projects that immediately work with their existing Google Docs, Google Calendars, Google Sites and Google Accounts (including OpenID). This dramatically reduces the barrier to the user trying a new app and increases relevance (since their existing data is available within the app).

We're presenting at Google I/O next week where we'll share more tips on this topic in our session, Reach new customers fast: Learn how to sell your cloud app on the Google Apps Marketplace. We’ll also be in the Developer Sandbox and would love for you to stop by our station and share any questions, tips or tricks.

Posted by Amit Kulkarni, Co-Founder and CEO of Manymoon

For those not attending Google I/O, remember to tune in to http://youtube.com/GoogleDevelopers on Wednesday, May 19, and Thursday, May 20, to watch the Google I/O keynote presentations live.

Keynote times:
Wednesday, May 19: 9:00am - 10:30am PDT
Thursday, May 20: 8:30am - 10:00am PDT

To stay up to date on I/O news, follow us on Twitter or Buzz — and to go one level deeper on I/O session content, live wave with us.

By Christine Tsai, Google Developer Team

Last week I spoke a test manager who told me with his shoulders hanging low, that his test department had grown substantially, but hardly had developed. He realized that he was confronted with his own limits on his knowledge, experience and to a certain extent on his own personality. The organization had expressed her satisfaction with the test department on more than one occasion, but this was not the feedback he had hoped for. He struggled for a time with the question “what is the next step?”, not knowing the answer.

It troubled him that his own organization didn’t have an answer to that question. “Am I ahead of the rest of the organization?”

Talking the matter over we realized that the thing which makes you ‘big’ also keeps you ‘small’. To break through this dilemma you have to break through your own believes, pick up unconventional approaches and start experimenting with new methods. In that respect a test manager should not only be concerned with the test process, but also with his own personal development.

Where to start? How do you address today’s problems and at the same time develop towards future needs….as an individual, department and as an organization? Or is this question too far beyond the realm of testing?

We took a big sip of our coffee, noticed it had become undrinkable…. and started to ravage our brains and established some shared guidelines which might be helpful:

• Accept some ambiguity in the test profession. For instance be independent in your professional attitude, but accept that sometimes the organization only hears you instead of listens to you.
• Testing will never be ahead of organization’s IT processes in practice but only in preaching. Keep preaching until it becomes (best) practice. Be focused on adding value instead of entering a believer versus non believer discussion about testing.
• Test management (of just ‘process’) improvement might help you to formulate a roadmap for ‘the next step’, but only if it strengthens the whole software development lifecycle (SDLC). You need to focus on other best practices as well. Involve the stakeholders of the SDLC. Suppliers of software for instance can be supported with generic test agreements (see TMap NEXT) or with witness tests.
• There is an interesting similarity between auditing and testing: independent. An auditor needs to be independent and adheres to professional ethics. An auditor might lose his official status when he crosses the line of independency too often (on the penalty of being no longer registered). Testers claim to be independent but lack such a register and professional ethics. Still it might be beneficial to incorporate (into a test policy) for instance the ten test principles from TestGoal. These principles support the test manager in coaching his testers but also sets a certain level of service the organization can and in the end will expect from testing.
• Invest in personal development and not only in developing your test management skills. The effectiveness of the test process often depends rather on the form (how) than on the content (what). Release advice and finding defects are only effective if content and form are well balanced. A lot of books on personal development are available, but these lack interaction. Some people argue that effectiveness of a message does not depend on how, but rather on who. There is a case: Your reputation precedes you. It might be wise to invest some time in for instance a 360 degree feedback from direct colleagues, or a formal assessment to test your own skills….or consider joining the club and start playing golf.

We could have come up with many more or different `guidelines`…and probably will. We started our contemplative moment with “What is the next step?”….We realized that in this case the question itself is more valuable than the answer will ever be. It helps you to stay ahead, personally and professionally.

The bottom line is that you contemplate about your profession and personal development!

Do you have any personal guidelines you would like to share? What do you do to break through organizational limits or personal boundaries when it comes to the test profession?

Many thanks to Otto Vos, he has written this nice post! Follow him at twitter.com/ottovos, motivate him to be more active at twitter.

Before starting any discussion around alternative energy, nuclear energy, any kind of energy, you need to watch Dan Nocera's talk. Not so much for the solution but for a description of the problem. Glen B. Alleman