The Google Code Blog

Monday, June 22, 2009

We've Moved!

We've moved the Google Code Blog to googlecode.blogspot.com. Your subscription should already be pulling from the new URL.

In case you're not getting posts from googlecode.blogspot.com in your RSS reader, you can subscribe to the new blog.

Posted by Mike Marchak, Google Code Team

Thursday, June 11, 2009

Google Technology User Groups

My favorite part about Google I/O is the dozens of interesting conversations with developers -- getting a first-hand look at the different things that they are doing with our technologies. That's the spirit of the Google Technology User Groups -- regular meetups where local developers can get together to network and discuss, demo, and hack on Google's many developer offerings.

From lightning talks in Mountain View, to App Engine hackathons in Tokyo, to lectures in Berlin, the GTUGs are a great place to meet fellow developers and learn (or teach) something new.

At Google I/O, there were many folks eager to bring the spirit of the conference back to their hometowns by starting up GTUGs of their own. Since the conference ended, our list of current GTUGs has grown to include this 'baby boomer' generation of chapters. The following are all new groups looking for members and starting to set up their first events.

If there's one near you, check it out! Let the organizers know you're interested; suggest topics for discussion and even offer to do a talk about your own experiences.

Europe

Paris GTUG - http://groups.google.com/group/paris-gtug
Hamburg GTUG - http://www.hamburg-gtug.org
GTUG Munich - http://gtug-muc.org
Istanbul GTUG - http://www.istanbul-gtug.org/
Polish GTUG - http://www.gtug.pl

North America

Tri-Valley California GTUG - http://groups.google.com/group/tv-gtug
Berkeley GTUG - http://www.meetup.com/Berkeley-GTUG/
San Diego GTUG - http://www.meetup.com/sd-gtug/
NYC GTUG - http://sites.google.com/site/nycgtug
New Jersey GTUG - http://nj-gtug.org/
Philly/Delaware GTUG - http://sites.google.com/site/phillygtug/
Boston GTUG - http://groups.google.com/group/boston-gtug
Denver GTUG - http://groups.google.com/group/denver-gtug
Twin Cities GTUG - tc-gtug.org
Austin GTUG - http://sites.google.com/site/austingtug/
Michigan GTUG - http://groups.google.com/group/mi-gtug
Utah GTUG - http://utahgtug.blogspot.com/
Laguna GTUG - www.laguna-gtug.org
Quebec GTUG - http://groups.google.com/group/gtug-quebec/?pli=1

South America
Chile GTUG - http://groups.google.com/group/gtug-cl
Argentina GTUG - http://groups.google.com/group/gtug-ar

Asia
Kuala Lumpur GTUG - http://sites.google.com/site/gtugkl/
Hyderabad GTUG - http://sites.google.com/site/hydgtug/

Also a big shout-out to our existing chapters:

Silicon Valley GTUG - http://www.meetup.com/sv-gtug (watch the organizers, Kevin and Van, talk about GTUGs at Google I/O)
Pune GTUG - http://pune-gtug.blogspot.com/
Chico GTUG http://www.chico-gtug.org
Berlin GTUG - http://www.berlin-gtug.org
Tokyo GTUG - http://tokyo-gtug.org/


View GTUGs in a larger map

Don't see a chapter near you? Start one! Join our GTUG managers mailing list. Other info at gtugs.org.

Wednesday, June 10, 2009

Google I/O Interactive Map: Now with videos + some Open Source goodness!

If you attended Google I/O 2009 a few weeks ago, you may have noticed a kiosk station on the 2nd and 3rd floors of Moscone West labelled 'Interactive Conference Map, powered by Google Maps'. The kiosk simply pointed to a JavaScript Maps API-based interactive map of the venue I created in my 20% time.

Now that all the I/O session videos and presentations are live, we took the opportunity to mash up the videos with our interactive conference map to provide developers with an alternate way to navigate through 80+ keynote and session videos, and bring the action at I/O to life virtually. For example, here are videos of sessions that took place in Room 1 (click the tabs for Wednesday and Thursday sessions). And here's where the keynote sessions took place. Check out where we filmed interviews with I/O sandbox developers on their apps, technical challenges and business best practices.


Now, hopefully you enjoyed using the map and are now thinking, "Cool, I want to do something like this for my next event!" (or your college campus, or such). If you are, then good news everyone, I've open sourced the interactive conference map and all relevant resources. Inside the project, you'll also find a how to article outlining the steps I went through to create the map.

If you attended I/O, then I hope you enjoyed it and had time to stop by the conference map kiosk! If not, no worries, just make sure to check out the open source project and see if you can use the code and/or techniques in your next mapping project!

Gmail for Mobile HTML5 Series: Suggestions for Better Performance

On April 7th, Google launched a new version of Gmail for mobile for iPhone and Android-powered devices. We shared the behind-the-scenes story through this blog and decided to share more of our learnings in a brief series of follow-up blog posts. This week, I'll talk about a few small things you can do to improve performance of your HTML5-based applications. Our focus here will be on performance bottlenecks related to the database and AppCache.

Optimizing Database Performance

There are hundreds of books written about optimizing SQL and database performance, so I won't bother to get into these details, but instead focus on things which are of particular interest for mobile HTML5 apps.

Problem: Creating and deleting tables is slow! It can take upwards of 200 ms to create or delete a table. This means a simple database schema with 10 tables can easily take 2-4 seconds (or more!) just to delete and recreate the tables. Since this often needs to be done at startup time, this really hurts your launch time.

Solution: Smart versioning and backwards compatible schema changes (whenever possible). A simple way of doing this is to have a VERSION table with a single row that includes the version number (e.g., 1.0). For backwards-compatible version changes, just update the number after the decimal (e.g., 1.1) and apply any updates to the schema. For changes that aren't backwards compatible, update the number before the decimal (e.g., 2.0) at which point you can drop all the tables and recreate them all. With a reasonable schema design to begin with, it should be very rare that a schema change is not backwards compatible and even if this happens every month or so, users should get to use your application 20, 30 even 100 times before they hit this startup delay again. If your schema changes very infrequently, a simple 1, 2, 3 versioning scheme will probably work fine; just make sure to only recreate the database when the version changes!

Problem: Queries are slow! Queries are faster than creates and updates, but they can still take 100ms-150ms to execute. It's not uncommon for traditional applications to execute dozens or even hundreds of queries at startup – on mobile this is not an option.

Solution: Defer and/or combine queries. Any queries that can be deferred from startup (or at any other significant point in the application) should be deferred until the data is absolutely needed. Adding 2-3 more queries on a user-driven operation can turn an action from appearing instantaneous to feeling unresponsive. Any queries that are performed at startup should be optimized to require as few hits to the database as possible. For example, if you're storing data about books and magazines, you could use the following two queries to get all the authors along with the number of books and magazine articles they've writen:

SELECT Author, COUNT(*) as NumArticles
FROM Magazines
GROUP BY Author
ORDER BY NumArticles;

SELECT Author, COUNT(*) as NumBooks
FROM Books
GROUP BY Author
ORDER BY NumBooks;


This will work fine, but the additional query will generally cost you about 100-200 ms over a different (albeit less pretty) query like:

SELECT Author, NumPublications, PubType
FROM (
SELECT Author, COUNT(*) as NumPublications, 'Magazine' as PubType, 0 as SortIndex
FROM Magazines
GROUP BY Author
UNION
SELECT Author, COUNT(*) as NumPublications, 'Book' as PubType, 1 as SortIndex
FROM Books
GROUP BY Author
)
ORDER BY SortIndex, NumPublications;

This will return all the entries we want, with the magazine entries first in increasing order of number of articles, followed by the book entries, in increasing order of the number of books. This is a toy example and there are clearly other ways of improving this, such as merging the Magazines and Books tables, but this type of scenario shows up all the time. There's always a trade-off between simplicity and speed when dealing with databases, but in the case of HTML5 on mobile, this trade-off is even more important.

Problem: Multiple updates is slow!

Solution: Use Triggers whenever possible. When the result of a database update requires updating other rows in the database, try to do it via SQL triggers. For example, let's say you have a table called Books listing all the books you own and another called Authors storing the names of all the authors of books you own. If you give a book away, you'll want to remove it from the Books table. However, if this was the only book you owned by that author, you would also want to remove the author from the Authors table. This can be done with two UPDATE statements, but a "better" way is to write a trigger that automatically deletes the author from the Authors table when the last book by this author is removed. This will execute faster and because triggers happen asynchronously in the background, it will have less of an impact on the UI than executing two statements. Here's an example of a simple trigger for this case:

CREATE TRIGGER IF NOT EXISTS RemoveAuthor
AFTER DELETE ON Books
BEGIN
DELETE FROM Authors
WHERE Author NOT IN
(SELECT Author
FROM Books);
END;
We'll get into more detail on triggers and how to use them in another performance post to come.

Optimizing AppCache Performance

Problem: Logging in is slow!

Solution: Avoid redirects to the login page. App-Cache is great because it can launch the application without needing to hit the network, which makes it much faster and allows you to launch offline. One problem you might encounter though, is that the application will launch and then you'll need to hit the network to get some data for the current user. At this point you'll have to check that the user is authenticated and it might turn out that they're not (e.g., their cookies might have expired or have been deleted). One option is to redirect the user to a login page somewhere, allow him to authenticate and then redirect him back to the application. Regardless of whether or not the login page is listed in the manifest, when it redirects back to your application, the entire application will reload. A nicer approach is for the application itself to display an authentication interface which sends the credentials and does the authentication seamlessly in the background. This will avoid any additional reloads of the application and makes everything feel faster and better integrated.

Problem: AppCache reloading causes my app to be slow!

Solution: List as few URLs in the manifest as possible. In a series of posts on code.google.com, we talked about the HTML5 AppCache manifest file. An important aspect of the manifest file is that when the version gets updated, all the URLs listed in the file are fetched again. This happens in the background while the user is using the application, but opening all these network connections and transferring all that data can cause the application to slow down considerably during this process. Try to setup your application so that all the resources can be fetched from as few URLs as possible to speed up the manifest download and minimize this effect. Of course you could also just never update your manifest version, but what's the point of having rapid development if you never make any changes?


That's a brief intro to some performance considerations when developing HTML5 applications. These are all issues that we ran into ourselves and have either fixed or are in the process of fixing in our application. I hope this helps you to avoid some of the issues we ran into and makes your application blazing fast!

We plan to write several more performance related posts in the future, but for now stay tuned for next post where we'll discuss the cache pattern for building offline capable web applications.



Previous posts from Gmail for Mobile HTML5 Series
HTML5 and Webkit pave the way for mobile web applications
Using AppCache to launch offline - Part 1
Using AppCache to launch offline - Part 2
Using AppCache to launch offline - Part 3
A Common API for Web Storage

Another Round of Deprecation Policies for Labs Graduates

We recently published deprecation policies for a number of APIs that graduated from Google Code Labs. They state how long we'll support each version from when it's deprecated or a newer version is introduced. It will be 3 years for most, but the time period varies a bit from product to product.
We still need to update the terms for a couple remaining graduates, but should have them all done within the next couple weeks.

Tuesday, June 9, 2009

Nicholas C. Zakas: Speed Up Your JavaScript

Nicholas C. Zakas delivers the seventh Web Exponents tech talk at Google. Nicholas is a JavaScript guru and author working at Yahoo!. Most recently we worked together on my next book, Even Faster Web Sites. Nicholas contributed the chapter on Writing Efficient JavaScript, containing much of the sage advice found in this talk. Check out his slides and watch the video.



Nicholas starts by asserting that users have a greater expectation that sites will be fast. Web developers need to do most of the heavy lifting to meet these expectations. Much of the slowness in today's web sites comes from JavaScript. In this talk, Nicholas gives advice in four main areas: scope management, data access, loops, and DOM.

Scope Management: When a symbol is accessed, the JavaScript engine has to walk the scope chain to find that symbol. The scope chain starts with local variables, and ends with global variables. Using more local variables and fewer global variables results in better performance. One way to move in this direction is to store a global as a local variable when it's referenced multiple times within a function. Avoiding with also helps, because that adds more layers to the scope chain. And make sure to use var when declaring local variables, otherwise they'll end up in the global space which means longer access times.

Data Access: In JavaScript, data is accessed four ways: as literals, variables, object properties, and array items. Literals and variables are the fastest to access, although the relative performance can vary across browsers. Similar to global variables, performance can be improved by creating local variables to hold object properties and array items that are referenced multiple times. Also, keep in mind that deeper object property and array item lookup (e.g., obj.name1.name2.name3) is slower.

Loops: Nicholas points out that for-in and for each loops should generally be avoided. Although they provide convenience, they perform poorly. The choices when it comes to loops are for, do-while, and while. All three perform about the same. The key to loops is optimizing what is performed at each iteration in the loop, and the number of iterations, especially paying attention to the previous two performance recommendations. The classic example here is storing an array's length as a local variable, as opposed to querying the array's length property on each iteration through a loop.

DOM: One of the primary areas for optimizing your web application's interaction with the DOM is how you handle HTMLCollection objects: document.images, document.forms, etc., as well as the results of calling getElementsByTagName() and getElementsByClassName(). As noted in the HTML spec, HTMLCollections "are assumed to be live meaning that they are automatically updated when the underlying document is changed." Any idea how long this code takes to execute?

var divs = document.getElementsByTagName("div");
for (var i=0; i < divs.length; i++) {
var div = document.createElement("div");
document.body.appendChild(div);
}

This code results in an infinite loop! Each time a div is appended to the document, the divs array is updated, incrementing the length so that the termination condition is never reached. It's best to think of HTMLCollections as live queries instead of arrays. Minimizing the number of times you access HTMLCollection properties (hint: copy length to a local variable) is a win. It can also be faster to copy the HTMLCollection into a regular array when the contents are accessed frequently (see the slides for a code sample).

Another area for improving DOM performance is reflow - when the browser computes the page's layout. This happens more frequently than you might think, especially for web applications with heavy use of DHTML. If you have code that makes significant layout changes, consider making the changes within a DocumentFragment or setting the className property to alter styles.

There is hope for a faster web as browsers come equipped with JIT compilers and native code generation. But the legacy of previous, slower browsers will be with us for quite a while longer. So hang in there. With evangelists like Nicholas in the lead, it's still possible to find your way to a fast, efficient web page.


Check out other blog posts and videos in the Web Exponents speaker series:

Google I/O: Session videos on building apps using the AJAX and Data APIs

One of the best things about attending Google I/O is the chance to meet developers who are using our APIs and interacting with Google technology in ways we could never imagine. Not only was it amazing to see exciting examples of apps built on the AJAX and Data APIs being demoed at the developer sandbox, but it was also interesting to meet other developers who are just starting to use many of our APIs for their specific needs and cool ideas. Hopefully, by making all of our sessions available for free to watch on your own time, many of you who are interested in Google's APIs will get a better understanding of the ways we are making our API offerings easier to use, more efficient and much more feature rich.

Big Announcements & More

One of the most exciting announcements at this year's I/O was the developer preview of Google Wave. After its introduction during the Day 2 keynote, there were three sessions devoted to the Google Wave APIs: Programming With and For Google Wave, Google Wave: Powered by GWT, and Google Wave: Under the hood. We hope you're as excited as we are, and can't wait to see how you use these tools.

Another new product announcement this year was Google Web Elements, which allow you to easily add your favorite Google products onto your own website. There are elements for Google News, Maps, Spreadsheets, YouTube and others, with more to come. Be sure to check out the Day 1 keynote for a complete introduction to the simple copy and paste power of Google Web Elements.

Keeping webmasters in mind, two sessions were all about optimizing your site for search. In one talk, Matt Cutts reviewed real sites that *you* submitted. talking through real-life issues that effect developers when it comes to optimizing their app for search. The other session focused on how to maximize your site, your content, and your application's exposure to search engines.

Javascript & Google AJAX APIs

The session on Custom Search Engines focused on helping your users search the sites and topics that are relevant to you. Nick Weininger discussed some of the ways to embed search and ads onto your site (including the new Custom Search element), then customize the look and feel of the results. Adobe was on hand to show how they're using Custom Search Engines to enhance their products and insert contextual search into the developer's programming workflow. We also announced the launch of the Custom Search gadget for Blogger which gives your blog's visitors the ability to search not just your posts, but web pages linked from your blog, your blog lists, and link lists.

In the session Implementing your Own Visualization Datasource, attendees learned about building a server-side data source compatible with the Google Visualization API, including hearing about the experience from a Salesforce.com expert. Itai Raz also gave a great session on using the Visualization API with GWT and treated the audience to advanced Javascript tricks such as wrapping visualizations as gadgets.

Ben Lisbakken's session detailed some advanced Javascript techniques and then delved into some of great tips and tracks he learned while creating the Code Playground, a tool which can help developers learn about and experiment with many of Google's APIs. Some of the highlights include increasing the security and performance of applications and learning why App Engine is so easy on which to develop.

Jon Kragh of VastRank showed off some neat ways he's Using AJAX APIs to Navigate User-Generated Content, including using Google Maps to display nearby colleges and translating reviews into the viewer's language. Also, Michael Thompson explored the idea of Building a Business with Google's free APIs using example Google Gadgets, Google Gadget Ads, Mapplets, and the Maps API.

Google Data APIs

Jeff Fisher and Jochen Hartmann spoke on the future direction of the YouTube API as it becomes increasingly social. They used two sample applications to demonstrate the use of the activity feeds as well as the new "SUP" feed that allows high traffic websites to monitor YouTube for activity in a scalable manner.

The session about writing monetizable YouTube apps focused on creating applications that allowed access to YouTube videos in creative ways. In the talk, Kuan Yong showed how to expertly navigate through the YouTube API terms of service in order to avoid business pitfalls so that developers can monetize their own apps.

Eric Bidelman and Anil Sabharwal discussed the Document List Data API in detail, highlighting common enterprise use cases such as sync, migration, sharing, and legal discovery. Partners Syncplicity, OffiSync, and gDocsBar showed off compelling demos.

In the talk on the evolution of the Google Data protocol, Sven Mawson outlined all of the new features in the Google Data APIs that will help in the creation of more efficient applications. Two of the new additions included a compact and customizable JSON output and the option to retrieve only the parts of a feed that you want using partial GET.

Monsur Hossain and Eric Bidelman showed how to build a read/write gadget using OAuth and the Google Data JavaScript library. They went through a step by step set of instructions that explained how to set up the gadget code, how to get a token using the OAuth proxy, and how to read and write data to Blogger using the JavaScript library inside of an iGoogle gadget.

Google Geo APIs

Mano Marks and Pamela Fox started with a grab bag session covering the vast spectrum of Geo APIs, discussing touring and HTML 5 in KML, the Sketchup Ruby API (with an awesome physics demo), driving directions (did you know you can solve the Traveling Salesman Problem in Javascript?), desktop AIR applications, reverse geocoding, user location, and monetization using the Maps Ad Unit and GoogleBar. Pamela finished by sneak previewing an upcoming feature in the Flash API: 3d perspective view.

In the session on performance tips for Maps API mashups, Marcelo Camelo announced Google Maps API v3, a latency-oriented rewrite of our popular JS Maps API. Also see Susannah Raub's more in-depth talk about Maps API v3. Then Pamela gave advice on how to load many markers (by using a lightweight marker class, clustering, or rendering a clickable tile layer) and on how to load many polys (by using a lightweight poly class, simplifying, encoding, or rendering tiles). Sascha Aickin, an engineer at Redfin, showed how they were able to display 500 housing results on their real estate search site by creating the "SuperMarker" class.

Mano and Keith presented various ways of hosting geo data on Google infrastructure: Google Base API, Google App Engine, and the just-released Google Maps data API. Jeffrey Sambells showed how ConnectorLocal used the API (and their own custom PHP wrapper) for storing user data.

On the same day as announcing better integration between the Google Earth and Google Maps JS APIs, Roman Nurik presented on advanced Earth API topics, and released a utility library for making that advanced stuff simple.